{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Neural Network\n",
    "\n",
    "In this tutorial, we'll create a simple neural network classifier in TensorFlow. The key advantage of this model over the [Linear Classifier](https://github.com/easy-tensorflow/easy-tensorflow/blob/master/3_Neural_Network/Tutorials/1_Neural_Network.ipynb) trained in the previous tutorial is that it can separate data which is __NOT__ linearly separable. We will implement this model for classifying images of hand-written digits from the so-called MNIST data-set.\n",
    "\n",
    "We assume that you have the basic knowledge over the concept and you are just interested in the __Tensorflow__ implementation of the Neural Nets. If you want to know more about the Neural Nets we suggest you to take [this](https://www.coursera.org/learn/machine-learning) amazing course on machine learning or check out the following tutorials:\n",
    "\n",
    " [Neural Networks Part 1: Setting up the Architecture](https://cs231n.github.io/neural-networks-1/)\n",
    " \n",
    " [Neural Networks Part 2: Setting up the Data and the Loss](https://cs231n.github.io/neural-networks-2/)\n",
    " \n",
    " [Neural Networks Part 3: Learning and Evaluation](https://cs231n.github.io/neural-networks-3/)\n",
    "\n",
    "The structure of the neural network that we're going to implement is as follows. Like before, we're using images of handw-ritten digits of the MNIST data which has 10 classes (i.e. digits from 0 to 9). The implemented network has 2 hidden layers: the first one with 200 hidden units (neurons) and the second one (also known as classifier layer) with 10 (number of classes) neurons.\n",
    "\n",
    "<img src=\"files/files/nn.png\">\n",
    "\n",
    "___Fig. 1-___ Sample Neural Network architecture with two layers implemented for classifying MNIST digits\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "## 0. Import the required libraries:\n",
    "We will start with importing the required Python libraries."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# imports\n",
    "import tensorflow as tf\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. Load the MNIST data\n",
    "\n",
    "For this tutorial we use the MNIST dataset. MNIST is a dataset of handwritten digits. If you are into machine learning, you might have heard of this dataset by now. MNIST is kind of benchmark of datasets for deep learning and is easily accesible through Tensorflow\n",
    "\n",
    "The dataset contains $55,000$ examples for training, $5,000$ examples for validation and $10,000$ examples for testing. The digits have been size-normalized and centered in a fixed-size image ($28\\times28$ pixels) with values from $0$ to $1$. For simplicity, each image has been flattened and converted to a 1-D numpy array of $784$ features ($28\\times28$).\n",
    "\n",
    "<img src=\"files/files/mnist.png\">\n",
    "\n",
    "\n",
    "If you want to know more about the MNIST dataset you can check __Yann Lecun__'s [website](http://yann.lecun.com/exdb/mnist/).\n",
    "\n",
    "### 1.1. Data dimension\n",
    "Here, we specify the dimensions of the images which will be used in several places in the code below. Defining these variables makes it easier (compared with using hard-coded number all throughout the code) to modify them later. Ideally these would be inferred from the data that has been read, but here we will just write the numbers.\n",
    "\n",
    "It's important to note that in a linear model, we have to flatten the input images into a vector. Here, each of the $28\\times28$ images are flattened into a $1\\times784$ vector. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "img_h = img_w = 28             # MNIST images are 28x28\n",
    "img_size_flat = img_h * img_w  # 28x28=784, the total number of pixels\n",
    "n_classes = 10                 # Number of classes, one class per digit"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.2. Helper functions to load the MNIST data\n",
    "\n",
    "In this section, we'll write the function which automatically loads the MNIST data and returns it in our desired shape and format. If you wanna learn more about loading your data, you may read our __How to Load Your Data in TensorFlow __ tutorial which explains all the available methods to load your own data; no matter how big it is. \n",
    "\n",
    "Here, we'll simply write a function (__`load_data`__) which has two modes: train (which loads the training and validation images and their corresponding labels) and test (which loads the test images and their corresponding labels). You can replace this function to use your own dataset. \n",
    "\n",
    "Other than a function for loading the images and corresponding labels, we define two more functions:\n",
    "\n",
    "1. __randomize__: which randomizes the order of images and their labels. This is important to make sure that the input images are sorted in a completely random order. Moreover, at the beginning of each __epoch__, we will re-randomize the order of data samples to make sure that the trained model is not sensitive to the order of data.\n",
    "\n",
    "2. __get_next_batch__: which only selects a few number of images determined by the batch_size variable (if you don't know why, read about Stochastic Gradient Method)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def load_data(mode='train'):\n",
    "    \"\"\"\n",
    "    Function to (download and) load the MNIST data\n",
    "    :param mode: train or test\n",
    "    :return: images and the corresponding labels\n",
    "    \"\"\"\n",
    "    from tensorflow.examples.tutorials.mnist import input_data\n",
    "    mnist = input_data.read_data_sets(\"MNIST_data/\", one_hot=True)\n",
    "    if mode == 'train':\n",
    "        x_train, y_train, x_valid, y_valid = mnist.train.images, mnist.train.labels, \\\n",
    "                                             mnist.validation.images, mnist.validation.labels\n",
    "        return x_train, y_train, x_valid, y_valid\n",
    "    elif mode == 'test':\n",
    "        x_test, y_test = mnist.test.images, mnist.test.labels\n",
    "    return x_test, y_test\n",
    "\n",
    "\n",
    "def randomize(x, y):\n",
    "    \"\"\" Randomizes the order of data samples and their corresponding labels\"\"\"\n",
    "    permutation = np.random.permutation(y.shape[0])\n",
    "    shuffled_x = x[permutation, :]\n",
    "    shuffled_y = y[permutation]\n",
    "    return shuffled_x, shuffled_y\n",
    "\n",
    "\n",
    "def get_next_batch(x, y, start, end):\n",
    "    x_batch = x[start:end]\n",
    "    y_batch = y[start:end]\n",
    "    return x_batch, y_batch"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.3. Load the data and display the sizes\n",
    "Now we can use the defined helper function in __train__ mode which loads the train and validation images and their corresponding labels. We'll also display their sizes:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Successfully downloaded train-images-idx3-ubyte.gz 9912422 bytes.\n",
      "Extracting MNIST_data/train-images-idx3-ubyte.gz\n",
      "Successfully downloaded train-labels-idx1-ubyte.gz 28881 bytes.\n",
      "Extracting MNIST_data/train-labels-idx1-ubyte.gz\n",
      "Successfully downloaded t10k-images-idx3-ubyte.gz 1648877 bytes.\n",
      "Extracting MNIST_data/t10k-images-idx3-ubyte.gz\n",
      "Successfully downloaded t10k-labels-idx1-ubyte.gz 4542 bytes.\n",
      "Extracting MNIST_data/t10k-labels-idx1-ubyte.gz\n",
      "Size of:\n",
      "- Training-set:\t\t55000\n",
      "- Validation-set:\t5000\n"
     ]
    }
   ],
   "source": [
    "# Load MNIST data\n",
    "x_train, y_train, x_valid, y_valid = load_data(mode='train')\n",
    "print(\"Size of:\")\n",
    "print(\"- Training-set:\\t\\t{}\".format(len(y_train)))\n",
    "print(\"- Validation-set:\\t{}\".format(len(y_valid)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To get a better sense of the data, let's checkout the shapes of the loaded arrays."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "x_train:\t(55000, 784)\n",
      "y_train:\t(55000, 10)\n",
      "x_train:\t(5000, 784)\n",
      "y_valid:\t(5000, 10)\n"
     ]
    }
   ],
   "source": [
    "print('x_train:\\t{}'.format(x_train.shape))\n",
    "print('y_train:\\t{}'.format(y_train.shape))\n",
    "print('x_train:\\t{}'.format(x_valid.shape))\n",
    "print('y_valid:\\t{}'.format(y_valid.shape))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As you can see, __`x_train`__ and __`x_valid`__ arrays contain $55000$ and $5000$ flattened images ( of size  $28\\times28=784$  values). __`y_train`__ and __`y_valid`__ contain the corresponding labels of the images in the training and validation set respectively. \n",
    "\n",
    "Based on the dimesnion of the arrays, for each image, we have 10 values as its label. Why? This technique is called __One-Hot Encoding__. This means the labels have been converted from a single number to a vector whose length equals the number of possible classes. All elements of the vector are zero except for the $i^{th}$ element which is one and means the class is $i$. For example, the One-Hot encoded labels for the first 5 images in the validation set are:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 0.,  0.,  0.,  0.,  0.,  1.,  0.,  0.,  0.,  0.],\n",
       "       [ 1.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.],\n",
       "       [ 0.,  0.,  0.,  0.,  1.,  0.,  0.,  0.,  0.,  0.],\n",
       "       [ 0.,  1.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.],\n",
       "       [ 0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  1.]])"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_valid[:5, :]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "where the $10$ values in each row represents the label assigned to that partiular image. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. Hyperparameters\n",
    "\n",
    "Here, we have about $55,000$ images in our training set. It takes a long time to calculate the gradient of the model using all these images. We therefore use __Stochastic Gradient Descent__ which only uses a small batch of images in each iteration of the optimizer. Let's define some of the terms usually used in this context:\n",
    "\n",
    "- __epoch__: one forward pass and one backward pass of __all__ the training examples.\n",
    "- __batch size__: the number of training examples in one forward/backward pass. The higher the batch size, the more memory space you'll need.\n",
    "- __iteration__: one forward pass and one backward pass of __one batch of images__ the training examples."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Hyper-parameters\n",
    "epochs = 10             # Total number of training epochs\n",
    "batch_size = 100        # Training batch size\n",
    "display_freq = 100      # Frequency of displaying the training results\n",
    "learning_rate = 0.001   # The optimization initial learning rate\n",
    "\n",
    "h1 = 200                # number of nodes in the 1st hidden layer"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Given the above definitions, each epoch consists of $55,000/100=550$ iterations."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. Helper functions for creating the network\n",
    "\n",
    "### 3.1. Helper functions for creating new variables\n",
    "\n",
    "As explained (and also illustrated in Fig. 1), we need to define two variables $\\mathbf{W}$ and $\\mathbf{b}$ to construt our linear model. These are generally called model parameters and as explained in our [Tensor Types](https://github.com/easy-tensorflow/easy-tensorflow/blob/master/1_TensorFlow_Basics/Tutorials/2_Tensor_Types.ipynb) tutorial, we use __Tensorflow Variables__ of proper size and initialization to define them.The following functions are written to be later used for generating the weight and bias variables of the desired shape:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# weight and bais wrappers\n",
    "def weight_variable(name, shape):\n",
    "    \"\"\"\n",
    "    Create a weight variable with appropriate initialization\n",
    "    :param name: weight name\n",
    "    :param shape: weight shape\n",
    "    :return: initialized weight variable\n",
    "    \"\"\"\n",
    "    initer = tf.truncated_normal_initializer(stddev=0.01)\n",
    "    return tf.get_variable('W_' + name,\n",
    "                           dtype=tf.float32,\n",
    "                           shape=shape,\n",
    "                           initializer=initer)\n",
    "\n",
    "\n",
    "def bias_variable(name, shape):\n",
    "    \"\"\"\n",
    "    Create a bias variable with appropriate initialization\n",
    "    :param name: bias variable name\n",
    "    :param shape: bias variable shape\n",
    "    :return: initialized bias variable\n",
    "    \"\"\"\n",
    "    initial = tf.constant(0., shape=shape, dtype=tf.float32)\n",
    "    return tf.get_variable('b_' + name,\n",
    "                           dtype=tf.float32,\n",
    "                           initializer=initial)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.2. Helper-function for creating a fully-connected layer\n",
    "\n",
    "Neural network consists of stacks of fully-connected (dense) layers. Having the weight ($\\mathbf{W}$) and bias ($\\mathbf{b}$) variables, a fully-connected layer is defined as $activation(\\mathbf{W}\\times \\mathbf{x} + \\mathbf{b})$. We define __`fc_layer`__ function as follows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def fc_layer(x, num_units, name, use_relu=True):\n",
    "    \"\"\"\n",
    "    Create a fully-connected layer\n",
    "    :param x: input from previous layer\n",
    "    :param num_units: number of hidden units in the fully-connected layer\n",
    "    :param name: layer name\n",
    "    :param use_relu: boolean to add ReLU non-linearity (or not)\n",
    "    :return: The output array\n",
    "    \"\"\"\n",
    "    in_dim = x.get_shape()[1]\n",
    "    W = weight_variable(name, shape=[in_dim, num_units])\n",
    "    b = bias_variable(name, [num_units])\n",
    "    layer = tf.matmul(x, W)\n",
    "    layer += b\n",
    "    if use_relu:\n",
    "        layer = tf.nn.relu(layer)\n",
    "    return layer"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4. Create the network graph\n",
    "\n",
    "Now that we have defined all the helped functions to create our model, we can create our network.\n",
    "\n",
    "### 4.1. Placeholders for the inputs (x) and corresponding labels (y)\n",
    "\n",
    "First we need to define the proper tensors to feed in the input values to our model. As explained in the [Tensor Types](https://github.com/easy-tensorflow/easy-tensorflow/blob/master/1_TensorFlow_Basics/Tutorials/2_Tensor_Types.ipynb) tutorial, placeholder variable is the suitable choice for the input images and corresponding labels. This allows us to change the inputs (images and labels) to the TensorFlow graph."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Create the graph for the linear model\n",
    "# Placeholders for inputs (x) and outputs(y)\n",
    "x = tf.placeholder(tf.float32, shape=[None, img_size_flat], name='X')\n",
    "y = tf.placeholder(tf.float32, shape=[None, n_classes], name='Y')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Placeholder __`x`__ is defined for the images; its data-type is set to __`float32`__ and the shape is set to __[None, img_size_flat]__, where __`None`__ means that the tensor may hold an arbitrary number of images with each image being a vector of length __`img_size_flat`__.\n",
    "\n",
    "\n",
    "Next we have __`y`__ which is the placeholder variable for the true labels associated with the images that were input in the placeholder variable __`x`__. The shape of this placeholder variable is __[None, num_classes]__ which means it may hold an arbitrary number of labels and each label is a vector of length __`num_classes`__ which is $10$ in this case."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4.2. Create the network layers\n",
    "\n",
    "\n",
    "After creating the proper input, we have to pass it to our model. Since we have a neural network, we can stack multiple fully-connected layers using __`fc_layer`__ method. Note that we will not use any activation function (`use_relu=False`) in the last layer. The reason is that we can use `tf.nn.softmax_cross_entropy_with_logits` to calculate the `loss`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Create a fully-connected layer with h1 nodes as hidden layer\n",
    "fc1 = fc_layer(x, h1, 'FC1', use_relu=True)\n",
    "# Create a fully-connected layer with n_classes nodes as output layer\n",
    "output_logits = fc_layer(fc1, n_classes, 'OUT', use_relu=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4.3. Define the loss function, optimizer, accuracy, and predicted class\n",
    "\n",
    "After creating the network, we have to calculate the loss and optimize it. Also, to evaluate our model, we have to calculate the `correct_prediction` and `accuracy`. We will also define `cls_prediction` to visualize our results."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From <ipython-input-12-61597c158d19>:2: softmax_cross_entropy_with_logits (from tensorflow.python.ops.nn_ops) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "\n",
      "Future major versions of TensorFlow will allow gradients to flow\n",
      "into the labels input on backprop by default.\n",
      "\n",
      "See tf.nn.softmax_cross_entropy_with_logits_v2.\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# Define the loss function, optimizer, and accuracy\n",
    "loss = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(labels=y, logits=output_logits), name='loss')\n",
    "optimizer = tf.train.AdamOptimizer(learning_rate=learning_rate, name='Adam-op').minimize(loss)\n",
    "correct_prediction = tf.equal(tf.argmax(output_logits, 1), tf.argmax(y, 1), name='correct_pred')\n",
    "accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32), name='accuracy')\n",
    "\n",
    "# Network predictions\n",
    "cls_prediction = tf.argmax(output_logits, axis=1, name='predictions')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4.4. Initialize all variables\n",
    "\n",
    "As explained in the [Tensor Types](https://github.com/easy-tensorflow/easy-tensorflow/blob/master/1_TensorFlow_Basics/Tutorials/2_Tensor_Types.ipynb) tutorial, we have to invoke a variable initializer operation to initialize all variables."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Create the op for initializing all variables\n",
    "init = tf.global_variables_initializer()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5. Train\n",
    "\n",
    "After creating the graph, it is time to train our model. To train the model, As explained in the [Graph_and_Session](https://github.com/easy-tensorflow/easy-tensorflow/blob/master/1_TensorFlow_Basics/Tutorials/1_Graph_and_Session.ipynb) tutorial, we have to create a session and run the graph in our session."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training epoch: 1\n",
      "iter   0:\t Loss=2.28,\tTraining Accuracy=58.0%\n",
      "iter 100:\t Loss=0.56,\tTraining Accuracy=84.0%\n",
      "iter 200:\t Loss=0.15,\tTraining Accuracy=95.0%\n",
      "iter 300:\t Loss=0.31,\tTraining Accuracy=92.0%\n",
      "iter 400:\t Loss=0.37,\tTraining Accuracy=91.0%\n",
      "iter 500:\t Loss=0.15,\tTraining Accuracy=96.0%\n",
      "---------------------------------------------------------\n",
      "Epoch: 1, validation loss: 0.24, validation accuracy: 93.4%\n",
      "---------------------------------------------------------\n",
      "Training epoch: 2\n",
      "iter   0:\t Loss=0.18,\tTraining Accuracy=95.0%\n",
      "iter 100:\t Loss=0.16,\tTraining Accuracy=94.0%\n",
      "iter 200:\t Loss=0.11,\tTraining Accuracy=97.0%\n",
      "iter 300:\t Loss=0.14,\tTraining Accuracy=95.0%\n",
      "iter 400:\t Loss=0.14,\tTraining Accuracy=96.0%\n",
      "iter 500:\t Loss=0.15,\tTraining Accuracy=95.0%\n",
      "---------------------------------------------------------\n",
      "Epoch: 2, validation loss: 0.16, validation accuracy: 95.6%\n",
      "---------------------------------------------------------\n",
      "Training epoch: 3\n",
      "iter   0:\t Loss=0.15,\tTraining Accuracy=95.0%\n",
      "iter 100:\t Loss=0.12,\tTraining Accuracy=97.0%\n",
      "iter 200:\t Loss=0.12,\tTraining Accuracy=96.0%\n",
      "iter 300:\t Loss=0.06,\tTraining Accuracy=99.0%\n",
      "iter 400:\t Loss=0.06,\tTraining Accuracy=99.0%\n",
      "iter 500:\t Loss=0.15,\tTraining Accuracy=98.0%\n",
      "---------------------------------------------------------\n",
      "Epoch: 3, validation loss: 0.13, validation accuracy: 95.8%\n",
      "---------------------------------------------------------\n",
      "Training epoch: 4\n",
      "iter   0:\t Loss=0.10,\tTraining Accuracy=97.0%\n",
      "iter 100:\t Loss=0.05,\tTraining Accuracy=100.0%\n",
      "iter 200:\t Loss=0.10,\tTraining Accuracy=97.0%\n",
      "iter 300:\t Loss=0.04,\tTraining Accuracy=100.0%\n",
      "iter 400:\t Loss=0.08,\tTraining Accuracy=97.0%\n",
      "iter 500:\t Loss=0.18,\tTraining Accuracy=96.0%\n",
      "---------------------------------------------------------\n",
      "Epoch: 4, validation loss: 0.11, validation accuracy: 96.3%\n",
      "---------------------------------------------------------\n",
      "Training epoch: 5\n",
      "iter   0:\t Loss=0.15,\tTraining Accuracy=96.0%\n",
      "iter 100:\t Loss=0.11,\tTraining Accuracy=95.0%\n",
      "iter 200:\t Loss=0.15,\tTraining Accuracy=99.0%\n",
      "iter 300:\t Loss=0.06,\tTraining Accuracy=99.0%\n",
      "iter 400:\t Loss=0.04,\tTraining Accuracy=99.0%\n",
      "iter 500:\t Loss=0.02,\tTraining Accuracy=100.0%\n",
      "---------------------------------------------------------\n",
      "Epoch: 5, validation loss: 0.10, validation accuracy: 97.1%\n",
      "---------------------------------------------------------\n",
      "Training epoch: 6\n",
      "iter   0:\t Loss=0.06,\tTraining Accuracy=99.0%\n",
      "iter 100:\t Loss=0.07,\tTraining Accuracy=98.0%\n",
      "iter 200:\t Loss=0.04,\tTraining Accuracy=100.0%\n",
      "iter 300:\t Loss=0.04,\tTraining Accuracy=98.0%\n",
      "iter 400:\t Loss=0.04,\tTraining Accuracy=100.0%\n",
      "iter 500:\t Loss=0.11,\tTraining Accuracy=97.0%\n",
      "---------------------------------------------------------\n",
      "Epoch: 6, validation loss: 0.09, validation accuracy: 96.9%\n",
      "---------------------------------------------------------\n",
      "Training epoch: 7\n",
      "iter   0:\t Loss=0.04,\tTraining Accuracy=99.0%\n",
      "iter 100:\t Loss=0.05,\tTraining Accuracy=98.0%\n",
      "iter 200:\t Loss=0.04,\tTraining Accuracy=99.0%\n",
      "iter 300:\t Loss=0.06,\tTraining Accuracy=98.0%\n",
      "iter 400:\t Loss=0.03,\tTraining Accuracy=99.0%\n",
      "iter 500:\t Loss=0.03,\tTraining Accuracy=100.0%\n",
      "---------------------------------------------------------\n",
      "Epoch: 7, validation loss: 0.09, validation accuracy: 97.2%\n",
      "---------------------------------------------------------\n",
      "Training epoch: 8\n",
      "iter   0:\t Loss=0.01,\tTraining Accuracy=100.0%\n",
      "iter 100:\t Loss=0.12,\tTraining Accuracy=99.0%\n",
      "iter 200:\t Loss=0.01,\tTraining Accuracy=100.0%\n",
      "iter 300:\t Loss=0.02,\tTraining Accuracy=100.0%\n",
      "iter 400:\t Loss=0.05,\tTraining Accuracy=98.0%\n",
      "iter 500:\t Loss=0.07,\tTraining Accuracy=98.0%\n",
      "---------------------------------------------------------\n",
      "Epoch: 8, validation loss: 0.09, validation accuracy: 97.6%\n",
      "---------------------------------------------------------\n",
      "Training epoch: 9\n",
      "iter   0:\t Loss=0.07,\tTraining Accuracy=98.0%\n",
      "iter 100:\t Loss=0.04,\tTraining Accuracy=98.0%\n",
      "iter 200:\t Loss=0.05,\tTraining Accuracy=98.0%\n",
      "iter 300:\t Loss=0.03,\tTraining Accuracy=99.0%\n",
      "iter 400:\t Loss=0.03,\tTraining Accuracy=100.0%\n",
      "iter 500:\t Loss=0.01,\tTraining Accuracy=100.0%\n",
      "---------------------------------------------------------\n",
      "Epoch: 9, validation loss: 0.08, validation accuracy: 97.4%\n",
      "---------------------------------------------------------\n",
      "Training epoch: 10\n",
      "iter   0:\t Loss=0.02,\tTraining Accuracy=99.0%\n",
      "iter 100:\t Loss=0.01,\tTraining Accuracy=100.0%\n",
      "iter 200:\t Loss=0.03,\tTraining Accuracy=98.0%\n",
      "iter 300:\t Loss=0.03,\tTraining Accuracy=98.0%\n",
      "iter 400:\t Loss=0.02,\tTraining Accuracy=99.0%\n",
      "iter 500:\t Loss=0.04,\tTraining Accuracy=99.0%\n",
      "---------------------------------------------------------\n",
      "Epoch: 10, validation loss: 0.08, validation accuracy: 97.5%\n",
      "---------------------------------------------------------\n"
     ]
    }
   ],
   "source": [
    "# Create an interactive session (to keep the session in the other cells)\n",
    "sess = tf.InteractiveSession()\n",
    "# Initialize all variables\n",
    "sess.run(init)\n",
    "# Number of training iterations in each epoch\n",
    "num_tr_iter = int(len(y_train) / batch_size)\n",
    "for epoch in range(epochs):\n",
    "    print('Training epoch: {}'.format(epoch + 1))\n",
    "    # Randomly shuffle the training data at the beginning of each epoch \n",
    "    x_train, y_train = randomize(x_train, y_train)\n",
    "    for iteration in range(num_tr_iter):\n",
    "        start = iteration * batch_size\n",
    "        end = (iteration + 1) * batch_size\n",
    "        x_batch, y_batch = get_next_batch(x_train, y_train, start, end)\n",
    "\n",
    "        # Run optimization op (backprop)\n",
    "        feed_dict_batch = {x: x_batch, y: y_batch}\n",
    "        sess.run(optimizer, feed_dict=feed_dict_batch)\n",
    "\n",
    "        if iteration % display_freq == 0:\n",
    "            # Calculate and display the batch loss and accuracy\n",
    "            loss_batch, acc_batch = sess.run([loss, accuracy],\n",
    "                                             feed_dict=feed_dict_batch)\n",
    "\n",
    "            print(\"iter {0:3d}:\\t Loss={1:.2f},\\tTraining Accuracy={2:.01%}\".\n",
    "                  format(iteration, loss_batch, acc_batch))\n",
    "\n",
    "    # Run validation after every epoch\n",
    "    feed_dict_valid = {x: x_valid[:1000], y: y_valid[:1000]}\n",
    "    loss_valid, acc_valid = sess.run([loss, accuracy], feed_dict=feed_dict_valid)\n",
    "    print('---------------------------------------------------------')\n",
    "    print(\"Epoch: {0}, validation loss: {1:.2f}, validation accuracy: {2:.01%}\".\n",
    "          format(epoch + 1, loss_valid, acc_valid))\n",
    "    print('---------------------------------------------------------')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 6. Test\n",
    "\n",
    "After the training is done, we have to test our model to see how good it performs on a new dataset. There are multiple approaches to for this purpose. We will use two different methods.\n",
    "\n",
    "## 6.1. Accuracy\n",
    "One way that we can evaluate our model is reporting the accuracy on the test set."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Extracting MNIST_data/train-images-idx3-ubyte.gz\n",
      "Extracting MNIST_data/train-labels-idx1-ubyte.gz\n",
      "Extracting MNIST_data/t10k-images-idx3-ubyte.gz\n",
      "Extracting MNIST_data/t10k-labels-idx1-ubyte.gz\n",
      "---------------------------------------------------------\n",
      "Test loss: 0.06, test accuracy: 97.4%\n",
      "---------------------------------------------------------\n"
     ]
    }
   ],
   "source": [
    "# Test the network after training\n",
    "# Accuracy\n",
    "x_test, y_test = load_data(mode='test')\n",
    "feed_dict_test = {x: x_test[:1000], y: y_test[:1000]}\n",
    "loss_test, acc_test = sess.run([loss, accuracy], feed_dict=feed_dict_test)\n",
    "print('---------------------------------------------------------')\n",
    "print(\"Test loss: {0:.2f}, test accuracy: {1:.01%}\".format(loss_test, acc_test))\n",
    "print('---------------------------------------------------------')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 6.2. plot some results\n",
    "Another way to evaluate the model is to visualize the input and the model results and compare them with the true label of the input. This is advantages in numerous ways. For example, even if you get a decent accuracy, when you plot the results, you might see all the samples have been classified in one class. Another example is when you plot, you can have a rough idea on which examples your model failed. Let's define the helper functions to plot some correct and missclassified examples.\n",
    "\n",
    "### 6.2.1 Helper functions for plotting the results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def plot_images(images, cls_true, cls_pred=None, title=None):\n",
    "    \"\"\"\n",
    "    Create figure with 3x3 sub-plots.\n",
    "    :param images: array of images to be plotted, (9, img_h*img_w)\n",
    "    :param cls_true: corresponding true labels (9,)\n",
    "    :param cls_pred: corresponding true labels (9,)\n",
    "    \"\"\"\n",
    "    fig, axes = plt.subplots(3, 3, figsize=(9, 9))\n",
    "    fig.subplots_adjust(hspace=0.3, wspace=0.3)\n",
    "    for i, ax in enumerate(axes.flat):\n",
    "        # Plot image.\n",
    "        ax.imshow(images[i].reshape(28, 28), cmap='binary')\n",
    "\n",
    "        # Show true and predicted classes.\n",
    "        if cls_pred is None:\n",
    "            ax_title = \"True: {0}\".format(cls_true[i])\n",
    "        else:\n",
    "            ax_title = \"True: {0}, Pred: {1}\".format(cls_true[i], cls_pred[i])\n",
    "\n",
    "        ax.set_title(ax_title)\n",
    "\n",
    "        # Remove ticks from the plot.\n",
    "        ax.set_xticks([])\n",
    "        ax.set_yticks([])\n",
    "\n",
    "    if title:\n",
    "        plt.suptitle(title, size=20)\n",
    "    plt.show(block=False)\n",
    "\n",
    "\n",
    "def plot_example_errors(images, cls_true, cls_pred, title=None):\n",
    "    \"\"\"\n",
    "    Function for plotting examples of images that have been mis-classified\n",
    "    :param images: array of all images, (#imgs, img_h*img_w)\n",
    "    :param cls_true: corresponding true labels, (#imgs,)\n",
    "    :param cls_pred: corresponding predicted labels, (#imgs,)\n",
    "    \"\"\"\n",
    "    # Negate the boolean array.\n",
    "    incorrect = np.logical_not(np.equal(cls_pred, cls_true))\n",
    "\n",
    "    # Get the images from the test-set that have been\n",
    "    # incorrectly classified.\n",
    "    incorrect_images = images[incorrect]\n",
    "\n",
    "    # Get the true and predicted classes for those images.\n",
    "    cls_pred = cls_pred[incorrect]\n",
    "    cls_true = cls_true[incorrect]\n",
    "\n",
    "    # Plot the first 9 images.\n",
    "    plot_images(images=incorrect_images[0:9],\n",
    "                cls_true=cls_true[0:9],\n",
    "                cls_pred=cls_pred[0:9],\n",
    "                title=title)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 6.2.2 Visualize correct and missclassified examples"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAggAAAI7CAYAAACJEmNgAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzt3XmcFNXV//HvAUFUFERwA2GMuBFUXIhiXHhUDOIa4r4EjUvELTE+ksSgolHEJeqjElGMYkRcwBXi9lODgriB4L4ioEQRWUQlKAj390fVJN19apjunl5nPu/Xq19DnbpVfWq403361u0qCyEIAAAgVbNyJwAAACoPBQIAAHAoEAAAgEOBAAAAHAoEAADgUCAAAACHAgFAo2dmvc0smNmQcucCVAsKBFQsM9vGzG40s7fMbImZLTezz8zsH2Z2spm1KneOxWRmE80s5wuVmNmJ8Zvh6h6zi5AygEZkjXInACQxs4skXayoiH1J0p2SvpW0kaTekm6TNFDSLmVKsRq8LunhOtZ9VcpEAFQfCgRUHDO7QNIlkj6VdEQI4eWENgdJOq/UuVWZGSGEIeVOAkB14hQDKoqZ1UgaImmFpH5JxYEkhRAmSOqbsP2RZvZ8fEpimZm9aWZ/NLM1E9rOjh/rmdm18b9X1J6nNrMh8XB8bzM71sxeNrNvM4fnzWxXMxtnZvPi0yCfmtktZrZpHcfYzswuj0+d/DvO9XUzG2Zm65hZTXxqYe+4feqpgYlZ/zKzZGa/i/f9QMK6/cxsZfx7XCslfpiZjTazD8xsafx7mWZm55iZe10xs1Hxc2xuZmeZ2Ttm9l38O7/AzCxud4SZvRLvc76Z3ZR0Kqn2d2Fmm5rZXXHbZXEOx+Z4/O3M7AozezfexxIze8bM9k9o2zI+xtfMbHH8/zfbzB4xs/1yeV6g0jGCgEpzkqQWku4NIby1uoYhhO9Tl81sqKQ/SlogaYyiUxIHSBoq6Wdm1ieEsCJjNy0lPSupnaSnJH0taVZGm/Mk9ZE0XtI/JbVJec6TJI2U9L2kRxWNemwp6RRJB5vZbiGET1Labx7vo4ukaZJuVlSobyXpXEkjFA3/XyLpxLjdJSm5zF7d7yQfIYRrzay3pP5mdkYI4a9xrhtLGi3pO0lHhRCWpWw2TNIqSS9L+pei38k+kv5PUk9JJ9TxdNcoOkU0XtHv+xBJl0tqaWaL4v0+LGmSot/5mZKaKzqdlGl9SVMU/b7ukNRW0pGS7jazjiGEq+s7djPrImmipJr4OZ+QtI6kgyQ9YWa/DiGMTNlklKRjJL0l6e+SlknaVNIeigrWp+t7TqBqhBB48KiYh6RnJAVJp+S4Xa94u08kbZwSX0PRm1GQdEHGNrPj+NOS1knY55B4/VJJOyas30rSckkfSeqYsW4fSSslPZQRfyHe5x8T9tdeUquU5YnRn2jOv8MT4+eYER9D0qNvxjYbKCpulknaQVHR8nS8n5MSnmOLhFgzRXNFgqRdM9aNiuOzU39Xit7UF8S/4y8lbZuybk1J7ygqvjbM2F+IH/dLapYS31zSovj/5Ucp8d5x+yEZ+5moqNA5OiPeNv79LZO0URxrE7edKql5wvFvUO6/Hx48CvkoewI8eKQ+4jeEkPkGlsV2I+PtTktYt1X8Zv1xRry2QNihjn3WFgjX1bH+unj9gXWsf0jSD5LWjZd3jttPT31TW80xNbRAWN3j+oTt9ojzfU/RqEuQNDrH594p3u6ijHhtgXBywja3x+suTVh3cbxu74x4iHPdfDX/bxenxFyBoKgQCpLG1nEsh8brz4iX14uXX5Bkpf7b4MGj1A9OMaDSWPwz16/37RT/fDZzRQjhAzObK2lzM2sbQkidwf+dpDfq2fcrdcR7xT/3NrOeCes3VDQ8vpWi0wm7xfEnQwir6nnOQrgzhHBito1DCJPN7GJJlyk6VfOhpNOT2prZBpLOl9RP0o8UDcun6ljH00xNiH0W/5yWsO5f8c9OCes+CSFkng6SosLqYkk71pFDrdr/vzaWfH2EDvHPbSUphPC1mY2XdLCkGfGcjUmSXg4h/Lue5wKqDgUCKs1nkrZR8hvC6tTOC/i8jvWfS+oct0stEOaHEOorRubVEd8g/nl+Pdu3jn+2jX/+q66GFeBBSZcqOl1wWwjh28wGZtZW0quKhvNfUXQufpGiT/RtJf1G0emBJEsSYj9ksa5Fwrov6niO2v+vNnWsr1X7/9cnftSldcq/j5L0e0nH6r9zQ74zs3GS/jeEUFdOQNXhWwyoNJPjn/vmuF3tm8vGdazfJKNdrWxGKupqU7uvNiEEW83jubhdbWFS16frsoq/LXBPvLhY0kVmtnVC01MUFQeXhBB2DSGcEUIYHKKvVN5XmmwlRdfESFLbB5IKjlS1639Tz//fSbUbhBCWhRCGhBC2UlRwHq+ozx4vaVwDjgWoOBQIqDR3KPqK4y/MrNvqGmZ8dXF6/LN3QruuikYkZmWcXmiol+Kfe+bY/mdJXwVMsFKSzKx5ronl6VpF5+WvkHS0pLUl3ZfwNcOu8U/3tUjFX80skc4WfS02U+/45/SEdaly/f9LE0L4NIRwt6SfKTods0d86gVoFCgQUFFCCLMVTTJrKekfZpZ4pUQz6yvp8ZTQ7fHPwWbWIaVdc0VfrWsm6W8FTvcmRcXMdWa2VUKOLc3sP28+IYRpir6W10PRMHVm+w0y3owXxj87FzTrBGb2C0VfJXxB0eS+pyRdpahguDaj+ez4Z++MfeyoaO5CqTSXdGVqsRV/jfQcRacmRq9u4xDCVEVzCPqb2a+S2pjZdma2YfzvDma2a0KzdSStGz/n8nwOBKhEzEFAxQkhDDWzNRRNNHvVzKYomtxWe6nlvRRda2BqyjZTzOwqSYMkvRWfE16q6DoI3RUNA9f7vfgc83wvfmO5XdLbZvaEpA8UnS/vrOiT6ZeK5lTUOl7RJLqh8ZvyREUTM7eUtH/cdnbc9hlJR0h60MweU/SVuzkhhLuyTLFHHZPvavMfIv3n4lS3KTqtcGwIYWXcZLCi3/VAM3smhFA7YvB3RfMurjez/1H06XlLRdcOeFDRefpSeEPSrpKmmdlTiuYcHKVoHsSgEMLMLPZxrKKJrX8zs3MUXdfhK0UjTtsr6ju9JM1XdGroJTN7V9Jrir4Wup6i495Y0g0hhG8Kd3hAmZX7axQ8eNT1UDR7/EZFF6X5WtGns88VjRycLGnNhG2OVlQMfKPoGwpvS/qTUq4vkNJ2tqTZq3n+IYrmH/SuJ8/tFH2Nb46i7+wvinO+RdI+Ce03kHSlpPfjHL9S9J37yyWtndKuuaKvG36saKQiSJqYxe/tRNX/NccQt22haKg9SOqfsK8uigqHr5TylUJJ3RRdGGq+okJsmqK5CTXxvkZl7GdUHK/J5fecciwnZsSDouJqU0UjBfPj3+VrioqczP30VsJ1EOJ160q6ID6GbxUVYrMk/UPSaYqvkaGo8LhIUUHxr/j/+vM4j2PEVx95NLKHhZDrt8kAoLziS1E/F0LoXe5cgMaKOQgAAMChQAAAAA4FAgAAcJiDAAAAHEYQAACAQ4EAAAAcCgQAAOBQIAAAAIcCAQAAOBQIAADAoUAAAAAOBQIAAHAoEAAAgEOBAAAAHAoEAADgUCAAAACHAgEAADgUCAAAwKFAAAAADgUCAABwKBAAAIBDgQAAABwKBAAA4FAgAAAAhwIBAAA4FAgAAMChQAAAAA4FAgAAcCgQAACAQ4EAAAAcCgQAAOBQIAAAAIcCAQAAOBQIVcTMRpvZkHLnARQKfRqNUWPp12UvEMzs25THKjNblrJ8XBnyeSojp+VmNj3LbU8xs5Xxdl+b2XQz61fsnOvI5cKM41gW57Z+OfJpSiqwT//BzN42s2/M7GMz+10O21ZSnz7EzKaY2Vdm9rmZ3WJmrcuRS1NUgf16XzObGPfLj3LctpL6dUczGx/36WBmncqRR5KyFwghhNa1D0mfSDo4JXZ3ZnszW6PI+eyfkdMrksbmsItJ8XbrS/q7pLFm1iazUQmO488Zx/EXSc+EEBYX83lReX06dryktpIOlHSumR2ew7YV0aclrSvpEkmbSPqxpM0lDSvycyJWgf16qaTbJP0+z+0rpV+vkvSYpFz+Jkui7AVCfczsMjO7z8zuMbNvJB2fOXxjZvuZ2eyU5U5m9pCZfWlms8zszDyfewtJvSTdleu2IYSVkm6XtLakzWtzNLMLzGyepJHxcxxiZq/Hn4omm1n3lOff2cxmxJ/87pG0Zp7HYZJOkHRnPtujsErdp0MIw0II00MIK0MI70oaL+mnueZd7j4dQrg7hPBkCGFZCGGRojeHnI8DxVGGfv1SCGG0pFkNybsC+vXnIYSbJU1ryHEUQ8UXCLGfSxojqY2k+1bX0MyaS5og6VVJHSX1kXS+me0br9/bzBZk+bwDJP0zhPBprgnHVefJkr6RNDMOd5LUWlJnSWeYWU9Fne8USRso6qSPmFlLM1tT0iNxrF3878NSjzPuqLtlkc7/KKqSH8r1OFA0ZenTZtZM0h6S3s414Qrr05K0Vz7HgaIq12t13iqwX1eMaikQJocQxocQVoUQltXTdjdJ64UQhoYQlocQPpL0N0lHS1II4bkQQvv6njDlU/eoHHPdw8y+kjRP0ZDRYSGEb+J1P0gaEue1TNJpkv4aQng1/nR3e9yup6JPRkHSjSGEFSGEeyX9Zy5E3L5tCOGlLHIaIOn+EMK/czwWFE/J+3Tsz4r64d9zyLXi+rSZHSDpWEkX53AcKL5y9et8VFy/rjSlOPdZCLl8gu8iqXP8H1+ruaSJOT7n3ooqxQdz3G5yCKF3Heu+CCEsT1nuIuk4Mzs3JdZSUTXdUtLcEEJIWTcnx1xkZutI+oWkA3LdFkVV8j5tZr9R9OK7Z0Y/rE+l9endFZ326x9CmFlfe5RUOV6r81VR/boSVUuBEDKWlyo6X1Rr45R/fyrpwxDCtg18zgGSxhX4U3fmcXwq6ZIQwpWZDeNhtszZrJ2V+5Dq4ZK+kDQ5x+1QXCXt02Z2mqTzJO0VQvgs3/0kKGmfNrNdJD0s6ZchhIm5pYoSKMdrdTGU47W64lTLKYZMMyQdaGbrm9kmks5JWfeipOVmdp6ZtYrP/2xnZjtnu3MzW1vRG+uohHWTzWxwA/OvdaukM82sp0Vam9nB8af+yZKamdlZZraGmR0haac8nmOApDszqltUnqL1aTMboGj2f58QwuyE9VXRp81sB0Wzvc8IITxWoHxRXMXs183MrJWkFtGitTKzFinrq6Jfx7m20n8nNq4Zz2sou2otEEZJelfRMM4Tku6tXRFC+EFSP0k/kTRb0gJJt0haT5LMrHfGkFaSX0j6UtKkhHWdJL3QoOz/m+vLkgZKulnSYkkfKPo6mkII3yua8HNqvK6/ok9Okv4z8eVbM+tV1/7NrLOiiVw5fwsDJTdKxevTlyk6XTbN/vu99ZtS1ldLn/7f+DhGpRzH64XIG0UzSsXr1/tIWibpUUk/iv/9eMr6qujX8STJZZJqj/UjRSMvZWd8sMyemdVIuiuEsGeZUwEKgj6Nxoh+XRgUCAAAwKnWUwwAAKCIKBAAAIBDgQAAAJycroPQvn37UFNTU6RUUO2mTZu2IITQodx55II+jdWhT6OxyaVP51Qg1NTUaOrUqfllhUbPzKru6mH0aawOfRqNTS59mlMMAADAoUAAAAAOBQIAAHAoEAAAgEOBAAAAHAoEAADgUCAAAACHAgEAADgUCAAAwKFAAAAADgUCAABwKBAAAIBDgQAAAJyc7uYIoPJcc801acvLli1zbd544w0XGzduXFb7HzhwoIv16tXLxU444YSs9gegOjCCAAAAHAoEAADgUCAAAACHOQhAFTnqqKNcbOzYsXnty8yyajdixAgXe/rpp11s7733Tlvu3LlzXnkB5fDBBx+42NZbb522fMMNN7g2Z599dtFyKjdGEAAAgEOBAAAAHAoEAADgUCAAAACHSYpAhSrkhMRtttnGxfr27etiH3/8sYs9+uijLvbRRx+52OjRo9OWL7jgglxSBMpq+vTpLtasWfpn6I4dO5YqnYrACAIAAHAoEAAAgEOBAAAAHAoEAADgMEkRqABTp051sYceeiirbbt37562nDSpsH379i7WunVrF1u+fLmL7brrri72+uuvu9jChQtXmydQyWbMmOFimX8j/fv3L1U6FYERBAAA4FAgAAAAhwIBAAA4FAgAAMCp+EmK48aNc7GRI0e62KabbupirVq1crHjjjsubXnjjTd2bbp27ZpLikCDff755y4WQnCxzAmJkvTkk0+mLW+yySZ553HNNde42LvvvpvVtgcddFDezwuU0ptvvuliN954o4v98pe/LEU6FYsRBAAA4FAgAAAAhwIBAAA4FAgAAMCp+EmK559/vovNnj077/2NGDEibXm99dZzbbp165b3/otts802c7FBgwa52C677FKKdFAgBx98sIsl3VJ53XXXdbF27doVLI/77rvPxZKurghUs/fff9/Fli5d6mJJt1xvShhBAAAADgUCAABwKBAAAIBDgQAAAJyKn6R42223uVjSrWaTJha+8847LjZ9+vS05YkTJ7o2L730kot17tzZxT755BMXy1aLFi3SlpNux5t0db2k3JImLjJJsfp16dKlqPu/+uqrXeyDDz7IatukW0AnxYBKdNVVV7lYTU2NizX111FGEAAAgEOBAAAAHAoEAADgUCAAAACn4icp7rvvvlnFkvTt27feNosXL3axzImMUvJklVdffTWrPJKsueaaactbb721a7PNNtu42KJFi1xsiy22yDsPNA0TJkxwsYsuusjFvv/+exfbaKONXGzYsGEutvbaa+eZHVA8SVfeTXrtTnoNXmeddYqRUtVgBAEAADgUCAAAwKFAAAAATsXPQSi29ddf38X22WefrLbNdi5ENh544AEXS5ofsf3227vY0UcfXbA80DhNnTrVxZLmGyRJuqPd3nvv3eCcgFJ47rnnsmrXoUOHImdSfRhBAAAADgUCAABwKBAAAIBDgQAAAJwmP0mxXObPn5+2fMYZZ7g2IQQXS7q4Tbt27QqXGKreYYcd5mJPPvlkVtsOGDDAxS677LIG5wSUyxtvvJFVu0GDBhU5k+rDCAIAAHAoEAAAgEOBAAAAHAoEAADgMEmxTIYPH562nDlpUZLatm3rYkl3HEPT9vnnn6ctT5kyxbVJumpi0pXjBg8e7GKtW7duQHZA6bz44osudscdd7jYjjvu6GJ9+vQpSk7VjBEEAADgUCAAAACHAgEAADgUCAAAwGGSYglMnjzZxYYNG1bvdo888oiLde/evSA5ofHo379/2vKCBQuy2u64445zsS222KIgOQHl8Mwzz7jY4sWLXaxv374u1qpVq6LkVM0YQQAAAA4FAgAAcCgQAACAQ4EAAAAcJimWwGOPPeZiy5cvT1veb7/9XJtevXoVLSdUp0cffdTFpk+fXu92vXv3drFLL720ECkBFeP111/Pqt0RRxxR5EwaB0YQAACAQ4EAAAAcCgQAAOBQIAAAAIdJigW2bNkyF3viiSdcbM0110xbvuSSS1ybFi1aFC4xVJ2FCxe62NChQ10sc8Jrkh49ergYt3FGNZs3b56LTZo0ycW22WYbF/v5z39elJwaG0YQAACAQ4EAAAAcCgQAAOBQIAAAAIdJigV29dVXu1jSle4OOOCAtOXdd9+9aDmhOv3lL39xsVdeeaXe7Q477DAX46qJaGxGjRrlYl988YWLZb7WInuMIAAAAIcCAQAAOBQIAADAYQ5CA0yYMMHF/vznP7tYmzZtXOzCCy8sSk5oPK699tq8ths+fLiLcVEkNDZz5szJqt36669f5EwaL0YQAACAQ4EAAAAcCgQAAOBQIAAAAIdJillKurPeOeec42I//PCDi/Xr18/FevXqVZjEgAxJfbXQdwZNmnib+RwrVqxwbZYsWZLV/hcvXuxi1113XZbZpWvevLmLXXnllS629tpr57V/lMf48eOzanfQQQcVOZPGixEEAADgUCAAAACHAgEAADgUCAAAwGGSYoKVK1e6WN++fV1s1qxZLta1a1cXS7q6IlAs22+/fdGf48gjj3SxTTbZJG056c569957b9FyysVGG23kYoMHDy5DJsjGpEmTXCypf6GwGEEAAAAOBQIAAHAoEAAAgEOBAAAAHCYpJpg5c6aLTZ06Nattk27Ru8UWWzQ4JzQ9SVfgfPjhh8uQiXf//fcXbF9JV3ls1iy7zy6HHHKIi+2yyy71brfHHntktX9UhoceesjFkq5au+OOO7rY3nvvXZScmgJGEAAAgEOBAAAAHAoEAADgUCAAAACnyU9SnDNnjovtv//+WW17zTXXuBi3FkWhPPjggy521VVXudjy5cvz2v8777zjYg250uHJJ5+cttylS5estvvFL37hYttuu23eeaD6/fvf/05bfvzxx7Pa7ogjjnCxpNt9IzuMIAAAAIcCAQAAOBQIAADAoUAAAABOk5+keMstt7hY0sTFJElX6DKzBucE1GXQoEFF3f+YMWOKun8gG5lX12zbtq1rc+ihh7rYb37zm6Ll1BQxggAAABwKBAAA4FAgAAAAhwIBAAA4TW6S4qRJk9KWb7rppjJlAgBIkjlJ8cUXXyxTJk0bIwgAAMChQAAAAA4FAgAAcJrcHITJkyenLX/zzTdZbde1a1cXa926dUFyAgCg0jCCAAAAHAoEAADgUCAAAACHAgEAADhNbpJiNnr06OFizzzzjIu1a9euFOkAAFByjCAAAACHAgEAADgUCAAAwKFAAAAATpObpPjHP/5xtcsAAIARBAAAkIACAQAAOBQIAADAoUAAAACOhRCyb2z2paQ5xUsHVa5LCKFDuZPIBX0a9aBPo7HJuk/nVCAAAICmgVMMAADAoUAAAAAOBQIAAHAoEAAAgEOBAAAAHAoEAADgUCAAAACHAgEAADgUCAAAwKFAAAAADgUCAABwKBAAAIBDgQAAABwKBAAA4FAgAAAAhwIBAAA4FAgAAMChQAAAAA4FAgAAcCgQAACAQ4EAAAAcCgQAAOBQIAAAAIcCAQAAOBQIVcTMJpvZieXOAygk+jUaGzMbbWZDyp1HQ5W9QDCzb1Meq8xsWcrycWXIp5mZXWNmi8xsoZldYWaW5baXmdmKOPevzOwFM9u12DmvJp/zzWyemS0xs9vMrGW5cmlqKq1fp+S1ppl9aGazc9imovp1Sl7PmVkodx5NRaX1aTPb18wmmtnXZvZRjtueYmYr49y/NrPpZtavWLnWk0tHMxtvZp+bWTCzTuXII0nZC4QQQuvah6RPJB2cErs7s72ZrVHklAZK6iepu6QekvpLOjmH7e+Oj2VDSS9LeiCpUbGPw8wOlHSepP+RtLmkrSVdVMznxH9VYL+u9QdJn+exXUX065TnGSApq8IdhVGBfXqppNsk/T7P7SfFx7K+pL9LGmtmbTIbleA4Vkl6TNLhRX6enJW9QKhP/OnlPjO7x8y+kXR85vCNme2X+onIzDqZ2UNm9qWZzTKzM3N4ygGSrgkhfBZC+FTStZJOzDXvEMJySXdK6mhmbeOK9Xkzu8HMFkkaHOd6ipm9Z2aLzexxM9ss5Tj6mtn78QjA/ym3F8QBkm4NIbwbQlgk6bJ8jgPFUYZ+LTPrKukoSVflm3cF9GuZ2fqS/qSo2EGFKHWfDiG8FEIYLWlWQ/IOIayUdLuktSVtXpujmV1gZvMkjYxzPcTMXo9H0SabWfeU49jZzGaY2Tdmdo+kNXN4/s9DCDdLmtaQ4yiGii8QYj+XNEZSG0n3ra6hmTWXNEHSq5I6Suoj6Xwz2zdev7eZLVjNLn4s6fWU5dfjWE7MbE1Fb8izQwhfxeHdJb0rqYOkK83scEnnSzo0jr2s6DhlZhtKGqfoRbC9pLmSdk3Z/+ZxR900h+PomFQho2xK2a8l6SZFn7a+yzfhCujXkjRM0o2S5ud7HCiaUvfpBotHCE6W9I2kmXG4k6TWkjpLOsPMeioqFE6RtIGiguIRM2sZ/008Esfaxf8+LPU44z69W7GPpdCqpUCYHEIYH0JYFUJYVk/b3SStF0IYGkJYHkL4SNLfJB0tSSGE50II7ZM2NDNTVEUuSQkvkbRuDrkea2ZfSfpU0naK/mBqfRJCuDmEsDI+jl9LGhpCeD+E8IOiT/k/MbOOkg6SNCOE8FAIYYWkv0j6snZHIYRZIYS2IYTP6sijdcJxKMdjQXGVpF9LkpkdIemHEML4PHOtiH5t0dyHnpL+mudxoLhK1qcLYI+4T89TNLx/WAjhm3jdD5KGxHktk3SapL+GEF6N+/ntcbuekn4qKUi6MYSwIoRwr6TptU8St28bQnipiMdSFKU679lQn+bQtoukzvF/fK3mkibWt2EIIZjZvyWtlxJeT1Flma0xIYQT61iXeRxdJA2Ph1lrrVJUvW6a2j6EsMrM5uaQx7fyx1EbR2UoSb82s9aSrpD0s5yyS1f2fm1mzRQVBmeHEFZadnOHUVol6dMFMjmE0LuOdV/Ep9NqdZF0nJmdmxJrqWjko6WkuSGE1AmzcwqaaZlUS4GQOVN5qaJP+rU2Tvn3p5I+DCFsm+dzvS1pB0mvxcs7xLFCyDyOTyVdGEJwQ3Hx+a2+KcvNFL3AZqv2OB6Ml3eQ9K+UYWGUX6n69TaKhkpfiN9UW0pqE59f7RnPtWmIUvXrdoomDj8QH0fzeB/zJPUPIUzJPXUUWClfq4spqU9fEkK4MrNhfEoksw93VuHeN8qmWk4xZJoh6UAzW9/MNpF0Tsq6FyUtN7PzzKxVfP5nOzPbOct9/13SeWa2qUVfNzlX0qjalWY218yOL9BxjJD0JzPbNt532/j8rRSdm+thZofG58jOVXQ+N1t/l3SqmW1jZu0UTR4bVaC8URzF6tczFL1g9Ygfv5b0Wfzvz6Sq6dcLFX1iqz2Og+N4D0lTC5Q7Cqtor9UWfSW9laQW0aK1MrMWKesnm9ngAh3HrZLONLOeFmltZgeb2TqSJktqZmZnmdka8em8nXLZeXwctRMb14znNZRdtRYIoxRNipoj6QlJ99auiM959pP0E0mzJS2QdIviIXaawT+fAAAfVklEQVQz650xpJXpr5KeVFT9vaFowsnf4m1bKfpKzMuFOIgQwlhF35IYa2Zfx8/3s3jdF4pmnF+t6IWxc+rzmtmPLPoOb+JkrhDCBEnXSXpe0e/hQ0mXFiJvFM0oFaFfhxB+CCHMq31IWixpZby8slr6dYikHseCOD4vYzgYlWOUivdavY+kZZIelfSj+N+Pp6zvJOmFQhxECOFlRV+Bv1nR388Hko6P132vaE7OqfG6/pIert02Lny+NbNeSfuOC+VlkmqP9SNFIy9lZ+mnTbA6ZtZb0skhhBPKnQtQKPRrNDZmViPprhDCnmVOpapRIAAAAKdaTzEAAIAiokAAAAAOBQIAAHByug5C+/btQ01NTZFSQbWbNm3aghBCLl/FLDv6NFaHPo3GJpc+nVOBUFNTo6lT+boxkplZ1V09jD6N1aFPo7HJpU9zigEAADgUCAAAwKFAAAAADgUCAABwKBAAAIBDgQAAABwKBAAA4FAgAAAAhwIBAAA4FAgAAMChQAAAAA4FAgAAcCgQAACAQ4EAAAAcCgQAAOBQIAAAAIcCAQAAOGuUO4FKtHTpUhc7//zzXWzEiBEutssuu7jY2LFjXaxLly55ZgcAQPExggAAABwKBAAA4FAgAAAAhwIBAAA4TFJM8Nlnn7nYyJEjXax58+YuNnXqVBcbP368i5111ll5Zgeke+2119KW+/fv79rMnj27RNms3lNPPeVi2267rYttttlmpUgH+I+k1+lDDjkkbfnGG290bQYOHOhiSe8N1YgRBAAA4FAgAAAAhwIBAAA4FAgAAMBp8pMUv/zySxcbMGBAGTIB8vPkk0+mLX///fdlyqR+jz76qIvdfvvtLnbvvfeWIh00UQsXLnSxpMmGmc4++2wXO/nkk11srbXWyi+xCsMIAgAAcCgQAACAQ4EAAAAcCgQAAOA0uUmKN9xwQ9ryww8/7Nq8+uqrBX3OSZMmuVgIIW15hx12cG322muvguaB6vfDDz+42GOPPVaGTPKTdDv0a6+91sWSbrm+zjrrFCUnND3PP/+8i/3rX/+qd7tjjjnGxVq1alWQnCoRIwgAAMChQAAAAA4FAgAAcCgQAACA0+QmKf72t79NWy7FbTkffPDBemOdO3d2be6//34X23nnnQuXGKrOP//5TxebMmVK2vLvf//7UqWTs0WLFrnY22+/7WL//ve/XYxJishH0pVFL7vssrz2dcIJJ7iYmeW1r2rACAIAAHAoEAAAgEOBAAAAHAoEAADgNOpJiv369XOxzCsYrly5sqDP2b59exdLmlw1Z86ctOVZs2a5Nj179nSxVatWNSA7VJM333zTxY4++mgX69q1a9ryBRdcULScGirpds9AMb3xxhsu9tprr2W17RprpL9FHnDAAQXJqVowggAAABwKBAAA4FAgAAAAhwIBAAA4jWaS4nPPPedi7733notlXvWqIVdSPP30011s//33d7E2bdq42LPPPpu2fPnll2f1nDfffLOLDRw4MKttUV2S+kTSFQZHjx6dtty6deui5ZSLpKsmJv2dNuYr0aH8kq5km60+ffoUMJPqwwgCAABwKBAAAIBDgQAAAJyqnIMwe/ZsF0u6gMyCBQvy2n/SnRUPP/xwF7v44otdbO21187qObp06ZK2fMstt7g2SfkPGjTIxb777jsXO+uss1ysRYsWWeWG0hs3bpyLPfbYYy6WeVEkKfmCWpUg6Y55SfMNevfu7WJt27YtRkpogpLmvSRp2bKliw0dOrTQ6VQVRhAAAIBDgQAAABwKBAAA4FAgAAAApyonKa5YscLF8p2QuNdee7nYfffd52JJd2lsiMxJikl34Pvd737nYkuXLnWxpImLhxxyiIttscUWuaSIEho7dqyLJf1fV/JFsTInD48ZM8a1ybw7niQNHjzYxZhQi3xMmTLFxV588cWstk2aYN6jR48G51TNGEEAAAAOBQIAAHAoEAAAgEOBAAAAnKqcpNgQmVedu+OOO1ybQk9IzEbSpMK7777bxV555ZVSpIMiWrJkiYu99NJLWW17xhlnFDqdgrn11lvTlr/88kvXplu3bi62zz77FC0nNC2vvvpq3ttW8gTgcmEEAQAAOBQIAADAoUAAAAAOBQIAAHAazSTFlStXZtXu5ZdfLnIm+QkhuNiqVauyapd07Em3oh49enSe2aGQvv/+exebO3euix1zzDGlSKdgZs6cWW+b7t27lyATNFXZTlJMup14JU8ALhdGEAAAgEOBAAAAHAoEAADgUCAAAACnKicpjhgxwsWaN29ehkwKZ/z48S42ffp0FzMzF0s69ksuuaQwiaHg1l13XRdLuq3sm2++6WKLFi1ysXbt2hUmsRzMnz/fxZJuWZ3ppz/9aTHSQRM0efJkF0u6xXiSNm3auFinTp0anFNjwwgCAABwKBAAAIBDgQAAABwKBAAA4FTlJMUJEyaUO4WcJN329p133klbHjp0aN77T7o9dYsWLfLeH4prrbXWcrGuXbu62Lhx41zswAMPdLHf/e53hUlM0ltvveViSVdInDNnjoslTaDN1KwZn0lQGAsXLnSxpCvNJunTp0+h02mU+GsFAAAOBQIAAHAoEAAAgEOBAAAAnKqcpFhtLr/8chcbPnx4XvuqqalxsTvvvNPFOnfunNf+UR5DhgxxsaQJV0kTdI8++uiC5dGhQwcXS5p8uGDBgrz2f9JJJ+W1HZApmyt3Ssm3dj7ttNMKnU6jxAgCAABwKBAAAIBDgQAAABzmIBRYv379XOy9994r2P67devmYnvuuWfB9o/y2HbbbV3s/vvvd7GkO3wmXcgoX4cffnhW7QYMGOBio0ePrne7pItEAfWZO3eui2V758akuzT27NmzwTk1BYwgAAAAhwIBAAA4FAgAAMChQAAAAE5VTlJMuoDMypUrs9r28ccfr7fNqaee6mKfffZZVvtPyi2bu9xlq9ruZInC2nHHHbOKFduPfvSjvLZ78803XWy77bZraDpo5KZMmeJi2d658dBDDy10Ok0GIwgAAMChQAAAAA4FAgAAcCgQAACAU5WTFAcOHOhigwYNymrbAw88MG25efPmWW2XbbukyZLZbpvp9NNPz2s7oNiSJohlM2mMCYnIx8KFC7Nq1759exf77W9/W+h0mgxGEAAAgEOBAAAAHAoEAADgUCAAAACnKicp9u/f38WuuuoqF1uwYEEp0qlX0sSZzNv7jhw50rXZZJNNipYT0BBJVwct5BVDgVRPPvlkVu0222wzF2vTpk2h02kyGEEAAAAOBQIAAHAoEAAAgEOBAAAAnKqcpNilSxcXu++++1zs4YcfdrHrr7++KDmtzp/+9CcXO+uss0qeB1Ao3333Xb1t1lprrRJkgsZoxYoVacsfffRRVtu1atXKxVq0aFGQnJoiRhAAAIBDgQAAABwKBAAA4FAgAAAApyonKSbZa6+9sortv//+acu33nqrazN+/HgXO/jgg13s17/+tYsl3fK2W7duLgZUszvuuMPF2rZtm7Z80UUXlSodNDLNmqV/du3Zs6dr8/bbb7vYlltuWbScmiJGEAAAgEOBAAAAHAoEAADgUCAAAACn0UxSzFbfvn1XuwygfkmTxs4999y05X322adU6aCRad68edry5Zdf7tok3V58p512KlpOTREjCAAAwKFAAAAADgUCAABwmtwcBAANl3QxMaBYNt10Uxe7/fbby5BJ08IIAgAAcCgQAACAQ4EAAAAcCgQAAOBQIAAAAIcCAQAAOBQIAADAoUAAAAAOBQIAAHAoEAAAgEOBAAAAHAoEAADgUCAAAACHAgEAADgWQsi+sdmXkuYULx1UuS4hhA7lTiIX9GnUgz6NxibrPp1TgQAAAJoGTjEAAACHAgEAADgUCAAAwKFAAAAADgUCAABwKBAAAIBDgQAAABwKBAAA4FAgAAAAhwIBAAA4FAgAAMChQAAAAA4FAgAAcCgQAACAQ4EAAAAcCgQAAOBQIAAAAIcCAQAAOBQIAADAoUAAAAAOBQIAAHAoEAAAgEOBAAAAHAqEKmJmk83sxHLnARSKmY02syHlzgMopMbyWl32AsHMvk15rDKzZSnLx5UxrzXN7EMzm53DNpeZ2Yo496/M7AUz27WIaWab13NmFsqdR1NRaX3azNY3s7vM7Eszm29mF+aw7SlmtjLO/Wszm25m/YqZ72pyMTO7yMw+iXMZY2aty5FLU1SB/Tr19bb20TmPbcv6Wp3xN1b72LMcuWQqe4EQQmhd+5D0iaSDU2J3Z7Y3szVKlNofJH2ex3Z3x8eyoaSXJT2Q1KhUx2FmAyRZKZ4LkQrs0zdIaiGps6TdJP3KzE7IYftJ8bGsL+nvksaaWZvMRiU4jl9JOlpSL0kdJa0n6f+K/JyIVWC/luLX25THJ7luq8p4rZ6UcRyTSvCc9Sp7gVCfuNK7z8zuMbNvJB2fOSxpZvulftI3s05m9lD8iWmWmZ2Z43N2lXSUpKvyzTuEsFzSnZI6mlnbuEp83sxuMLNFkgbHz3WKmb1nZovN7HEz2ywlj75m9r6ZLTGz/1OOb/Rmtr6kPykqdlAhytCnD5J0ZQhhWQjhY0l3KHqzzUkIYaWk2yWtLWnz2hzN7AIzmydpZJzrIWb2evzJbLKZdU85jp3NbIaZfWNm90haM4cUDpY0MoTwrxDCN4r+Po8xs1a5HgsKrxyv1YVQCa/VlariC4TYzyWNkdRG0n2ra2hmzSVNkPSqok8ZfSSdb2b7xuv3NrMF9TzfTZJ+L+m7fBM2szUlnShpdgjhqzi8u6R3JXWQdKWZHS7pfEmHxrGXFR2nzGxDSeMUvbm3lzRX0q4p+988fgHedDVpDJN0o6T5+R4HiqaUfdqU/oJlkrrX0XZ1eawh6WRJ30iaGYc7SWqtaHTiDDPrqahQOEXSBooKikfMrGX8N/FIHGsX//uw1OOM+/RuORzHWpK2yPVYUDSlfq0+zMwWmdlbZvbrfBKukNfqXcxsQVxk/Cn+3ZRdtRQIk0MI40MIq0IIy+ppu5uk9UIIQ0MIy0MIH0n6m6KhSYUQngshtK9rYzM7QtIPIYTxeeZ6rJl9JelTSdsp+oOp9UkI4eYQwsr4OH4taWgI4f0Qwg+SLpP0EzPrqOhT34wQwkMhhBWS/iLpy9odhRBmhRDahhA+q+M4dpXUU9Jf8zwOFFfJ+rSkJyT9wcxam9mWil4M184h1z3iPj1P0uGSDos/wUvSD5KGxHktk3SapL+GEF6N+/ntcbuekn4qKUi6MYSwIoRwr6TptU8St28bQnipjjwel3SamXUxs7aSBsXxXI4FxVXKfn2PpG0VvWGfLunS+PU7WxXxWi3pn4oK9g0lHSHpBEm/y+E4iqZU5/Mb6tMc2naR1Dn+j6/VXNLE+ja0aMLTFZJ+llN26caEEE6sY13mcXSRNDwekqq1StGnsk1T24cQVpnZ3GwSMLNmigqDs0MIK80axWhXY1OSPh07S9FI0kxFL1z3SPpFDs8/OYTQu451X8RDtKm5Hmdm56bEWir6hNhS0twQQuqE2Tk55DFS0d/G84o+3FwnqZ+iT2yoDCXr1yGEt1MWJ5vZjYoK2LFZPn/ZX6vj9jNTFt8ws8sknS3p6mz3USzVUiBkzsBfqvRPDRun/PtTSR+GELbN43m2UTRU+kL8ptpSUpv4/GrPEEIunT9J5nF8KunCEIIbiovP2/ZNWW6mqDNmo52kHpIeiI+jebyPeZL6hxCm5J46CqxUfVohhAWSjqldNrOrJL2Sz76Sdp+x/KmkS0IIV2Y2jIeOM/twZ0lvZ7ZNfKJoDsRg/feccL/4+eblmDOKp2T9uo7nLtSnoVK9Vtf13BXxqa5aTjFkmiHpQIu+vrWJpHNS1r0oabmZnWdmreLzmtuZ2c5Z7rezojfXHoqGlT6L//2ZJJnZXDM7vkDHMULSn8xs23jfbeNzXVJ0bq6HmR0an/s9V9FQWjYWKvrEVnscB8fxHpKmFih3FFax+rTMrKuZtTOzNczsQEXzCC5PWT/ZzAYX6DhulXSmmfW0SGszO9jM1pE0WVIzMzsrzuUISTtlu2Mza29mP4r3213SNYpOb/AV3spVzH59WPyaafEp1bMUzWupXV8Nr9UyswPieQwys26KJpY/svqtSqNaC4RRiiaQzFF0fvXe2hXx+aF+kn4iabakBZJuUfSVKJlZ74whLaVuG0KYV/uQtFjSynh5pUWzpddXNEGlwUIIYyVdq+hrY19LekPx6Y0QwheKvklxtaI3/M6pzxu/UH6bNPElRFKPY0Ecn5cxHIzKMUpF6NOxnoo+pX8t6c+SjgohvJeyvpOkFwpxECGElyUNlHSzor+fDyQdH6/7XtF53lPjdf0lPVy7bfwG8a2Z9apj9x0U/W6WKnpRviVljgMq0ygVr18fK+ljRZNm75R0We3XLavltTq2v6S3zGyppPGS7pfkRuDKwSi+s2dmvSWdHELI5TvkQMUysxpJd4UQKuLCLEAh8FpdGBQIAADAqdZTDAAAoIgoEAAAgEOBAAAAnJyug9C+fftQU1NTpFRQ7aZNm7YghJD113sqAX0aq0OfRmOTS5/OqUCoqanR1Kl8jR7JzCyXq+JVBPo0Voc+jcYmlz7NKQYAAOBQIAAAAIcCAQAAOBQIAADAoUAAAAAOBQIAAHAoEAAAgEOBAAAAHAoEAADgUCAAAACHAgEAADgUCAAAwKFAAAAADgUCAABwKBAAAIBDgQAAABwKBAAA4KxR7gQAAKhEixcvTlv+5JNP8t5Xly5dXOy6665zse7du7vYVlttlba8ww475J1HLhhBAAAADgUCAABwKBAAAIBDgQAAAJxGPUlx/vz5LnbkkUemLe++++6uzWmnneZiNTU1Bcur0JYsWeJizz//vIv17dvXxVq0aFGUnACgUk2YMMHFxo8f72ITJ05MW/7www/zfs6tt97axWbPnu1i33//fb37WrVqVd555IIRBAAA4FAgAAAAhwIBAAA4FAgAAMBpNJMUM694JUk//vGPXSxzQt9GG23k2lTbhMSddtrJxRYsWOBiU6dOdbEtt9yyMImhYnz99dcu9oc//MHF3n77bRd7+umnXYyJrKhEM2fOdLHhw4e72K233upiy5Ytc7EQQmESq8P7779f1P0XAyMIAADAoUAAAAAOBQIAAHAoEAAAgFOVkxSTJuBlXiFRkhYuXOhiZ555ZtryjTfeWLjESuCyyy5zsVmzZrlY0sQcJiQ2TqNHj05bHjx4sGuT7W1qkyY4brDBBvklBhTR3LlzXez6668vQybeNtts42JJt3GudIwgAAAAhwIBAAA4FAgAAMChQAAAAE5VTlJ87bXXXCzztpx1ueiiiwqcTfG89dZbLnbNNde42M9//nMXO+qoo4qSE8oraWLWueeem7acNInXzLLa/9lnn+1iN910k4u1a9cuq/0BqZL6ZtLEwj322CNtOelW9S1btnSxNm3auFjr1q1d7Ntvv3Wxn/3sZy6WObFw1113dW123HFHF1trrbVcbJ111nGxSscIAgAAcCgQAACAQ4EAAAAcCgQAAOBU/CTF+fPnu9gDDzyQ1ba33367i3Xo0KHBORVD0oTEPn36ZLVt//79XWzddddtcE6oPEmTVJOuGJqve++918Uef/xxF0u6WmPmBMekSWRoOpYuXepiSa9pr7/+uos9/PDD9e6/V69eLjZ9+nQXq6mpcbGkK4t26tTJxZo1a9qfoZv20QMAgEQUCAAAwKFAAAAADgUCAABwKn6S4nnnnedimbe3laSddtrJxY444oii5FQMkydPdrF58+a52EknneRixx9/fFFyQnnNmTPHxe644456t9thhx1cbKONNnKx//f//l9WeSxZssTFkiZLHnfccWnLG2+8cVb7R/Vbvny5ix177LEuljQh8YILLnCx/fbbL688kiYkJuncuXNe+29qGEEAAAAOBQIAAHAoEAAAgFPxcxCS7kKXFOvYsaOLVcqFWpYtW+ZiQ4cOTVsePny4a5N0nEkXf0LjNGPGDBf7+uuvXWyvvfZKW37uuedcm++++87FxowZ42JXXHGFi3300UculjQ/5tBDD01bTrrAEneBrH5Jd0LMfD2TpPHjx7tY0oXqzj//fBdbe+2188wOhcQIAgAAcCgQAACAQ4EAAAAcCgQAAOBU/CTFbE2YMMHF9t9/fxdr27Zt2vLAgQMLmsfEiROzir300kv17quaLvSEwvv+++9dLGni6rnnnlvvvlq1auViv/rVr1xs3LhxLjZz5kwXCyG4WObEskqZJIzCSrrT4rBhw1ysS5cuLjZp0iQXa9OmTWESQ8ExggAAABwKBAAA4FAgAAAAhwIBAAA4FT9J8Te/+Y2LPfvssy722WefuVjSFeUyJ1c98sgjDcjOS5q8lTSxLNMWW2zhYklXJ0PTcc8992TV7h//+Efa8mGHHZb3c06dOjXvbXfbbbe05datW+e9L1SuKVOmZNVuxx13dLFOnToVOh0UESMIAADAoUAAAAAOBQIAAHAoEAAAgFPxkxR33nlnF3vzzTddLOnWuE888YSLXXXVVWnLG264oWszYMCAXFJMc8IJJ7jY9ttvX+92u+++u4slTVxE03HMMce4WNKk2ldffTVt+b333nNtkv5mHnroIRdbvHixi2VefbSudrfeemvactLfQrdu3VwM1SXpaptJkm73fckll7jYIYcc4mJJExxReowgAAAAhwIBAAA4FAgAAMChQAAAAI4lXfmvLrvssktoyJXWmoKPP/7YxZImG/bo0SNt+amnnnJtOnToULjESsDMpoUQdil3Hrmo5D69aNEiF0vqS0uWLElbzvdqnpLUp08fFxs+fLiLHXTQQS72wQcfpC2fdtpprs2IESOyyqNS0Ke9pL6Ubf9K0rx5cxc7/fTT05Z33XVX1+bTTz91sa5du7rYj3/846zyePvtt12sV69eacuN4UqQufRpRhAAAIBDgQAAABwKBAAA4FAgAAAAp+KvpFhtLr30UhdLmsCTeUXHapuQiOJr166di40dO9bFDj/88LTlzEmLUvLExXPOOcfFrrzyShdr1aqVi/Xv39/FrrjiirTlJ5980rWZOXOmi3HF0Oryv//7vy72l7/8Je/9rVy50sUyJ8YmTZQthcwr7fbu3du1uffee0uUTekxggAAABwKBAAA4FAgAAAAhwIBAAA4TFJsgKQJY3feeaeLrbfeei62wQYbFCUnNG777befi2XefnfMmDGuTdItm5Mm1CZNSExy4YUXuti7776btpx0a+qk50z6m0HlGjZsmIsdeeSRLnbccce52IoVK1xs7ty5LpY0cbEc5s+fn7ac9JrfvXt3Fxs8eHDRciolRhAAAIBDgQAAABwKBAAA4DAHoQEef/zxrNodeOCBLrbTTjsVOh00UZnzEpLmKRTaWmut5WJHHXVU2nLSHIR//vOfLpZ018qki0ShMiTdfbFnz54ulnl3z7o888wzLpY5V2HIkCGuzSuvvJLV/gsp6YJj06ZNK3kepcIIAgAAcCgQAACAQ4EAAAAcCgQAAOAwSbEBkiYprrPOOi6WdPczoLHJvFjOo48+6tok3fnupptucrGLLrqocImhou277771tpkxY4aLJU1SbNGihYuddNJJLnbqqae62HXXXediSRcda0oYQQAAAA4FAgAAcCgQAACAQ4EAAAAcJilmacSIES42b948F9too41cjKsmoilo1iz988agQYNcm4cfftjFkq6Sd/TRR7vYVlttlX9yqGr777+/i11wwQUulnS3yFtvvdXFPvzwQxebOHFiXrl17Ngxr+2qASMIAADAoUAAAAAOBQIAAHAoEAAAgMMkxSwlTVI0Mxfr169fVvv75ptv0pYXL17s2nTu3DnL7IDK06NHDxf785//7GJJVxr94x//6GKjR492saTbTqPx2XbbbV0s8/biknTfffdltb+k244nWWON9LfIAw880LW58sors9pXNWIEAQAAOBQIAADAoUAAAAAOBQIAAHCYpFhgmZNapOTJVZm3Fu3evbtrc+eddxYuMaAC/PKXv3SxW265xcUefPBBF0u6+t32229fmMRQ0ZImo15//fUuljn5W5KmTZvmYl988YWL1dTUuFhmf0266mdjxggCAABwKBAAAIBDgQAAABwKBAAA4DBJscBGjhzpYrfddpuLnXLKKWnLF154YdFyAipFhw4dXOzpp592sS5durjYsGHDXGzMmDGFSQxVZ6ONNnKxCRMmuNhdd93lYi+++KKLJU1A3HDDDfNLrpFgBAEAADgUCAAAwKFAAAAADgUCAABwmKSYpRtvvNHFLr74Yhfba6+9XGzgwIEutv7666ctt2zZsgHZAdUr6bbmffr0cbFHH33Uxd555x0X69atW2ESQ6NwwgknZBWDxwgCAABwKBAAAIBDgQAAABzmIGRpzz33dLFnn322DJkAjd+4ceNcbIcddnCxjz76yMWYgwAUBiMIAADAoUAAAAAOBQIAAHAoEAAAgMMkRQAVZ7311nOxWbNmlSEToOliBAEAADgUCAAAwKFAAAAADgUCAABwKBAAAIBDgQAAABwKBAAA4FAgAAAAhwIBAAA4FkLIvrHZl5LmFC8dVLkuIYQO5U4iF/Rp1IM+jcYm6z6dU4EAAACaBk4xAAAAhwIBAAA4FAgAAMChQAAAAA4FAgAAcCgQAACAQ4EAAAAcCgQAAOBQIAAAAOf/Az1MPs5st6mbAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0xf038048>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAggAAAI7CAYAAACJEmNgAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzs3XecVPXVx/HvQSkqggUVEQGJlWhsoCYa62NsYMVHY0WwRuwlGjVqVCyJ0ViDsWBDCSp2QY2FYNQAYmJHo4ioIOQRQUTqef64d5OZOXd3Z3Znd2eXz/v12hfcM7+598zyY+bM7565Y+4uAACAXK2aOgEAAFB5KBAAAEBAgQAAAAIKBAAAEFAgAACAgAIBAAAEFAhoVGb2kpk1ymdrzWyKmU1pjGOVwsx2NjM3s0sybuttZs+Z2ax0zJtpfFi63aMB87okPcbODXWMls7MBqS/wwFNnQtQXxQIKFr6xOdmttTMflDDuBdzxg5oxBSbNTPrIOkpSdtIelDSpZL+2KRJ1SCnoKjp56WmzhNA3Szf1Amg2VmsZN4MkvSrwhvNbANJO+WMK3SUpBUbMsFm4O+SNpE0qyC+jaQ1JV3g7kMKbjtf0lWSPm/49Er2sqSXqrltSuOlAaCcKBBQqhmSvpR0jJn92t0XF9x+rCST9KSk/Qvv7O5TGz7Fyubu30l6P+OmLumfX2Tc50slv/dK9JK7X9LUSQAoL04xoC7+JKmzpL65QTNrLeloSX+T9E7WHbN6ECxxtJn9zcxmmtn3ZvaZmY0xs0My9tHVzG4wsw/Tsf9nZn83s4tqS9zMOprZOWb2gplNM7OF6TEfN7PtqrnPT83siXT8AjObbmavmdnFBePWMrPfmdkHZjbPzGanfx9mZj1zxuX1IJhZj/R3cnc65K7CUzQ19SCY2bZm9lCa18L0dzfUzLoUjk3Hb21mo81srpnNMbPnzezHtf3u6iv9N3MzuzbjtkHpbc+ZWauc+AAze9jMPjaz+Wm+r5jZEdUc46V0P63N7Ndm9q90jrxvZsfljDvRzN5K9znNzC7NPW46pke6r2FmtrGZPZrOtXlmNs7Mflbi4+9qZjelj2WBmf07nXd9MsaubGYXmdnb6WOemz6WEWa2dSnHBeqKFQTUxQOSfq9kteDRnPi+ktaSdJ6k9UvY3xVKltA/kfRnSd9IWltSH0kHSxpRNdDMeksaI2k1SWMlPaLklEUvSZdIuqyWY22SHm+skvP9X0vqlua+l5n1c/fROcfbMx03R9LjSpb4V0v38wslfQIysxUlvSLpB5Kek/SEkpWU7pL2k/SQpI+ryWl2up8t0rGPSXozve3Nau5Tld8xSgq2BWl+n0naQMm/TT8z2y531cbMfiLpeUltlPzuPkqP+5KkF2o6VhmcLeknks4wsxfc/ak0p16SblCyOnWEuy/Nuc+tkt5V8u/1paTVJe0t6V4z28jdqysKH5S0raSnJS2S1F/SbWa2SNKPlBSyT0r6i5J/+19L+k7S1Rn7Wk/Sq5LeljRUydw8RNIzZnaYu4/IuE8eM9tK0rNK5s4YJb/7TkpW2caZ2QHu/nQ61iSNTn9Xr0q6Xckpu3Ul7Szpr5Im1nZMoN7cnR9+ivqR5JKmpX+vetLqmnP7aCUv7itKujwdP6BgHy8l0y4v9m9J0yStmHHMTjl/b6OkiHBJh2WMXbdge4qkKQWxjrn7zIl3VbK0/15B/OH0eJvXklu/dNx1GePaSFo5Z3vndOwlBeMGZP3O0tuGpbf1yIltKGmhkhf5dQrG7yppiaRROTFTcmrDJe1XMP60NO6Sdi5yPlySjn8p/XvWz3YF91lfSbE1U9I6klZQ8sK7RNL/ZBzjB9X8Pv+i5IW/8HG/lOY0XtIqOfGe6e/q63QOrZNz2ypK+kFmSlo+J94j53fy24Lj9E6P/7WkDjX9Gyp5I/aRpO8l7VSwny5Kis4vJbVNY5ul+xiV8dhbSVq1If5/88NP4Q+nGFBXf5K0nKSBkmRm3SXtLul+T86xl2qRkheJPO6e28jXT8mT9uPuPjxj7Ge1HcTdvynYZ1V8mpJ3+RubWbeMu86vJbeaxi1097m15VYHJ0lqLek0d89rXnT3F5SsKPQzs5XT8E8kbSRprLs/VrCvmyT9q4557CTp4mp+8k7buPtHko5X8u55uKSbJf1Q0pXu/nzhjt095OTuC9P7LS9pt2pyOs/dZ+fc52NJ45QUA5fl/r7ScU+kOa2Tsa9vJP2mIIcJku5P93dANTlU2UfJytKN7v5ywX6+kHSNklN2hY8lay4tdfevazkeUBacYkCduPvrZvaWpIFmdrmSJe1WSgqHUt0v6RRJ75jZSCVd8a+6+zcF46pebJ6pY9qSJDPbXsk75h8r+dRAm4Ih60iqWpa/X9KBkl43sxGSXpT0SlpQ5HpZyTvB89Ll5KeVnHJ4091D4VMmVX0DO2Wdx1by2JZTstIwUdJWObnmcfclZjZOyQtZqS71EpoU3f1BM9tNyZzZUckL98VZY9Ni7ZdKXjy7KVlxyJX1gi5JEzJiVc2fWcvzVQVDV0mfFtz2RjUF3ktKTlVsqf/2j2Sp+nfqbhnXvlBySkhKTls9reSUypuSfp4W3o8p+R1NSIsjoFFQIKA+/qTk3PGeko6RNNHdJ9VhP2coefc6UEn/wnmSFpvZ05LOSt91Ssm7NakeH/UzswOUrBR8r6RX4F+S5klaqmTpfydJbavGu/sjZtZX0llpfiek+5ko6Xx3fy4dNydtcrxUyTntPdJdzDKzWyRd7u6L6pp3NVZP/zynlnHt0z87pn/OqGbc9HpnVLyHlBQIUvLOOhRRaWPn3yWtquS8+7NK3s0vUbKSdLRy/q1yZRSXUnJKTOk+qrutdcZttf2+OlZze5Wqf6eDaxnXXvpPsbarkr6I/vpvX8RcM7tbybz7tpZ9AfVGgYD6uFfJk9dQJe/kflPz8Gzpi8MfJP3BzNaUtIOkQ5U8of7QzH7o7guUNPNJ1b9rLMZlSs5F93b393JvMLOhSgqEwvyekvSUma2kpPGtr5Ll/SfNbEt3fzcdN03SoLTJrJeSPoCTlTzRt5JU66csSlT1QtfR3eeUMH6tam7vXP+UamdmnSTdoaQpUJKuN7MX3X1mwdAzlby4HuPuwwr28XMlBUJjqO33lVVw5Kq6fT93f7yYA6anEc5Q0tC5vpJ5eYKkwUoK5SOL2Q9QH/QgoM7Sc7cPKVmWnafk0w313edX7v6Iu/+vkq76H0jaNL35tfTPvepxiPUlvZtRHLRSUpjUlNs8d3/B3c+UNETJqYmQiyfecfcblfRlSBnXhCiDqt/HT4sc/0b6ZyiCzGw51fL4yyEtnoYpKfJOS3/WlnRPeluuqk/CPJyxq/AYGtBWOX0cuXZO/6xt1azUf6c87v6Ru9+h5DF/q+STLkCDo0BAfV2opElrj7o04plZWzPbrfDFwZJrKqyWbla903xCyScT9k3fQRbuq5iVhSmSNsi9RkB67IuVvOsv3OduZlZ43lv677vK79Jxm1r29yTkjSuzm5Q0d15nZhsW3mhmbcws90Xpb5I+kLSjmRW+yAxW3foPSnWmkqa9P7v77e5+u5KPJO6peKpkSvrnzrlBM9tD/z090Rg6KlkFys2ht6TDlawOjKrl/o8pOZV1spntnTXAzH6cflRWZraemf0wY9iqSk6phOZFoCFwigH14sln7OtzdcQVlHwuf4qZva6kQaydknfemyj5xMJ76bEWmtnBSs5FDzezE5S8O2uXjt1Ntc/p65R8v8EkM3tYyQvs9kqKgyeUfFIi17WSeljynQJTlJye2FrJ6YNPlby4SdL/SPq9mf1NyUcJv1KysrKfkv6G35bySymGu79vZgMl3amkwXO0pMlKzqN3U/KOdaakjdPxbmaDlPRePGxmVddB2DzNf7SSF+pS7VxN850kzXb36yUpbaS8UsnHDI/PGXOCkmteXGFmY9296h33LUp6W0am/1afK1lN2lPJ9TLCRbQayFhJx5rZtkoaT6uug9BK0gm1nd5x90VmdqCS6x88lc6RN5UUjesqeew90/1+p+TfY1Ta5/K2kubKNZTMpdbKvlYDUHYUCGhq85R0qe+i5GN4+0uaq+Qd10lKXvz+w90nmNkWShoZ90rvM1fJC11mJ3zB/Yea2QJJpys5hz1fSQPcMZIOUiwQhihZIemt5EV0qZKCaIik63M+cjZG0vVKuvL3k9RByWfbn5P0e3f/W1G/jRK5+31m9g8lTZS7SPqZkt/pF0pO/4woGP9Kuqpwhf57euR1Je/S91DdCoSdVP2S/6dKegw65uRyaG4TYdrgeaiSF98HzWwLd5/t7v80s12UXFNjbyXPV/9Q8qmS2Wq8AuETSScq+S6ME5W8i39D0m/cfUwxO0gfy+ZKVlD6KplvS5XMkUlK5m7Vx2YnKCmkdlLy77GqkkJvoqQb3L1en+IBimXujfLNuwDQrKSnjD6RdLe7D2jSZIAmQA8CAAAIKBAAAEBAgQAAAAJ6EAAAQMAKAgAACCgQAABAQIEAAAACCgQAABBQIAAAgIACAQAABBQIAAAgoEAAAAABBQIAAAgoEAAAQECBAAAAAgoEAAAQUCAAAICAAgEAAAQUCAAAIKBAAAAAAQUCAAAIKBAAAEBAgQAAAAIKBAAAEFAgAACAgAIBAAAEFAgAACCgQAAAAAEFAgAACCgQAABAQIEAAAACCgQAABBQIAAAgIACoRkxs/vM7JKmzgMoJ+Y1WpqWMqebvEAws29zfpaa2fyc7cObIJ/zzOwdM5trZh+b2Zkl3PdYM1uS5j7HzCaZ2d4NmW8NuZiZ/drMpqa5DDez9k2Ry7Ko0uZ1mlNvM/trmsN0Mxtc5P0qaV7/T/r7/Lapf5/Lmkqb02a2qpnda2YzzewrM7uohPtWzJxO8zndzKakufzdzH7SVLnkavICwd3bV/1ImiqpX07s/sLxZrZ8I6R1hKRVJO0j6Qwz61/Cff+aPpZVJd0jaaSZdSwc1AiPY6CkQyX9WNI6kjpI+kMDHxOpSpvXZrampKcl3SJpNUkbSnq+hF1UyryWpKm5v9+s3yfKr9LmtKQbJLWW1E3SdpIGmtmRJdy/Iua0mW0v6TJJByh53blX0iNmZg153GI0eYFQGzO73MxGmNkDZjZX0hGFyzfpu4opOdtdzWxUWll+YmYnF3s8d7/K3Se5+xJ3f0/SE5K2LzVvd18i6U5JK0parypHM/uVmU2X9Kc0133N7B9mNtvMxpnZpjmPY2szezNdzXhAUtsSUugn6U/u/rm7z5V0jaSfm1m7Uh8Lyq+x57WksyU95e4PuPtCd5/j7u+XmncFzGtUqCaY030lXe3u8939Y0l3KXljVJIKmNM9JL2Vvu4sVVKsrCWpU6mPpdwqvkBIHSBpuKSOkkbUNNDMlpP0pKTxSt457y7pHDPbLb19JzObVcxBzayVpB0kvVNqwmnVOUjSXEn/SsNdJbVXUvH+wsz6KJl8x0paXckkfczM2phZW0mPpbHV0r/vn/s404m6XXUppD+52ytI+kGpjwUNpjHn9XaSZpvZa5Ysxz5mZl1LTbgC5rUkrW1mMyw5BXitma1Y6uNAg2nMOZ31HLdpNWNryqOp5/RTktqZWZ/0dzJQ0kR3n1nqYym35lIgjHP3J9x9qbvPr2XsdpI6uPuQ9J3SR5LuULLcLnd/2d2Lrcwuk7RYSUVXrB3MbLak6ZL6S9o/fQevdF+XpHnNl3S8pFvcfXy6YnFnOq6PklULl3Sjuy9y9wclTao6SDp+FXd/rZo8npF0vJl1N7NVJJ2bxnkyrRyNOa+7Sjpa0slKnvQ+l1TK0nylzOt3JG0haW0lLyjbSfptCY8DDasx5/RoSeeZWXsz20DSAJX2/FYpc3qOpEck/U3SAknnp8drco1xvrAcPithbHdJ3dJ/+CrLSXqplAOa2WlKJupP3X1hCXcd5+47V3PbjIJ9dZd0uJmdkRNro6SabiNpmrt7zm2flpDHn5S8KIxVUgheJ2lvSdNK2AcaVmPO6/mS/uLuEyXJzC6VNN3M2rv7t0XcvyLmtbt/KenLdPNfZvZLSQ8rKXzQ9BpzTg+WdKOSd/0zJT0g6aASjl8Rc1rSCZKOlNRL0seS9pT0tJlt7u4zSthP2TWXFQQv2J6n/Eqxc87fP5P0YVqxVf2s7O79ij2YmR0v6SxJu7n7F3XOOip8HJ9JurQg1xXd/c9KngQLl4C7FX2gpGq90N27u/u6kt5Pjze9Pg8AZdWY8/qfGccr3K6rRpvX1Ry7yZu58B+NNqfdfZa7/9zd13L3TZW84f17vbLP2X3BdkPO6c0lPe7uH6bP209JmqWkwbxJNZcCodCbkvax5GMua0s6Nee2VyUtNLOzzKxdev5nMzPbupgdm9nRki6VtLu7T8m4fZyZXViGxyBJt0k6OT33ZOlSWT8zW0nSOEmtzGywmS1vZgdL2qrYHZtZJzPrme53U0m/U7JkVq4XBZRfg81rJQ1c/c3sR2bWWtIFkl6uWj1oRvN6FzNbN/17N0lXKjnni8rUkM/V65vZauk82kdJH8EVObc3izmtpAejr5n1SPe9h5JesZJ738qtuRYIwyS9p2QZZ7SkB6tucPfFSpbSt5E0RUklNlTJx/xkZjsXLGkVulxJE8pE++9nfG/Kub2rpFfK8SDc/XVJJ0m6VdLXkiYr+Yil3H2Bkoaf49LbDpT0aNV90/9M35pZdVXmGkp+N/OUNAINzTlvhso0TA00r939WUm/VtKb8pWSJdMjcoY0l3ndW9JrZvadkifmNySdUc1YNL1harjn6j5KXkTnKOkXO8TzP5nTXOb0XUp6EMamj+U6SYPc/cNy5F4fxhvK4plZD0n3uvtPmzgVoGyY12hpmNPlQYEAAACC5nqKAQAANCAKBAAAEFAgAACAoKQLJXXq1Ml79OjRQKmguZs4ceIsd1+jqfMoBXMaNWFOo6UpZU6XVCD06NFDEyZMqFtWaPHMrJSrh1UE5jRqwpxGS1PKnOYUAwAACCgQAABAQIEAAAACCgQAABBQIAAAgIACAQAABBQIAAAgoEAAAAABBQIAAAgoEAAAQECBAAAAAgoEAAAQUCAAAICAAgEAAAQUCAAAIKBAAAAAAQUCAAAIlm/qBAAsOxYvXhxiEyZMCLFJkyYVNe6DDz4IsY022ihve/DgwWHMlltuWWOeAFhBAAAAGSgQAABAQIEAAAACCgQAABDQpJghq5HKzEJsueWWa4x0gGZh0aJFedvjx48PY373u9+F2KhRo8qaxyuvvJK3/cYbb4QxWU2QqBxffPFF3vatt94axgwfPjzEPv7446L2f8wxx4TYrrvuGmKHHHJI3nbr1q2L2n9LwQoCAAAIKBAAAEBAgQAAAAIKBAAAECzzTYpPPvlkiB155JEh1qlTpxD71a9+FWJHH310iLVqRR2GliXrCoannXZa3vaYMWPKesys/4ObbbZZrfe76aabypoH6m7p0qUhltVseMUVV+Rtv//++2XN46677ioqVpjHX/7ylzCmS5cu5UuswvDKBQAAAgoEAAAQUCAAAICAAgEAAATLfJPipptuGmIDBw4MsYceeijEBg0aFGJZjS633357iG244YbFpgg0mqyriF500UUhdvPNN4fY3Llza91/x44dQ+ykk04KsZ///Ochtuaaa4ZY586daz0mKkfWc+EJJ5xQ6/06dOgQYkcddVSIrb/++kXl8eGHH4bY0KFDQ6ywOfLMM88MY+6///4QaylX2WUFAQAABBQIAAAgoEAAAAABBQIAAAiW+SbFHj16hNi1115bVOzZZ58NscKvB5WkrbfeOsQKm1/WWWedmtIEGsX5558fYllf0VyMPfbYo6h9ZTUKo/l78MEHQ+y6664r6r4bbbRR3vYzzzwTxqy33np1S6waO+20U4gVXh10xIgRYcw111wTYt26dStfYk2IFQQAABBQIAAAgIACAQAABBQIAAAgWOabFOvjZz/7WYhlXSnsf//3f0PsrbfeytumSRENKesKiRdccEGIFduQ2Lp16xAbPHhw3nbhV+VK0gorrFDU/tH8jR07NsSyvrY562qYTz/9dN52uRsSsxx88MEhdumll+Ztf/nllw2eRyVhBQEAAAQUCAAAIKBAAAAAAQUCAAAIaFIss4MOOijEsr7aedKkSXnbe+65Z4PlBGQ1JGZdAS5L9+7dQ+ziiy8OsWOOOab0xLDMO/LII0OsZ8+eTZAJCrGCAAAAAgoEAAAQUCAAAICgRfcgfP755yFW+G1cL7/8chiz6qqrhlj//v1DbIcddgixadOmhdi8efNC7IADDggxoC4KL4JUn29kbNOmTYhlfSvfdtttV2R2QM2yLpRUKQ488MC87XfeeSeMefjhh0PsjDPOaLCcGhMrCAAAIKBAAAAAAQUCAAAIKBAAAEDQLJsU586dG2IjR44MsaxGEXfP21599dXDmCVLloTY3XffHWJZzTXdunULsW233TbENt544xAD6uKee+7J2y62ITHL888/H2I0JKIhDR06NMTOPPPMJsgkevvtt2sd8+677zZCJk2DFQQAABBQIAAAgIACAQAABBQIAAAgaJZNii+++GKIjRkzJsROP/30EBs0aFDedlZT4cKFC4va/7777hti06dPD7EOHTqE2Keffpq3nfWNeUChcePGhVgxDV2tW7cOsVtvvTXEsq4OCjSkL774IsTGjh2bt73jjjs2Vjp5Jk+e3CTHrRSsIAAAgIACAQAABBQIAAAgoEAAAABBxTcpZjVlHXnkkSF2//33h1jfvn3rdMysr7z99ttvi7rveuutF2KFDTdSvLriY489VusYLFsKr/opSTfddFOIffPNN7Xua6WVVgqxBQsWhNh3330XYiussEKItWrFewvUbODAgSE2atSoEMtq7B48eHCN25K00UYbFZXHJ598EmJZrxdZCpvJs2Tln9XonvW6Uun4Xw4AAAIKBAAAEFAgAACAgAIBAAAEltUIVZ3evXv7hAkTGjCdKOurZrOaq7IaAVdeeeU6HfOZZ54JscIrMErSWmutFWKjR48Osddffz3ETjvttLztrEaXSZMmhVglf020mU10995NnUcpmmJOF2vevHkh1r59+0bP45RTTgmx888/P8TWXnvtxkinUTGny2vIkCEhdsEFFzRBJg3ryy+/DLHOnTs3QSZRKXOaFQQAABBQIAAAgIACAQAABBQIAAAgqPgrKa6zzjohtssuu4RYXRsSp06dGmLHHXdciLVr1y7EspoZsxoXs74WulevXnnbu+++exiz//77h9izzz4bYllfWY3m76KLLmrqFCRJN954Y4g9//zzIVb4NexZ/xewbDvrrLNCbIsttgixW265JW/7o48+CmPWX3/9EPv73/9eVB7bbLNNiGU977/11lt525Xyf7KxsIIAAAACCgQAABBQIAAAgIACAQAABBXfpHjeeeeF2MEHHxxi3bt3D7F+/fqF2JNPPpm3ffbZZ4cxWc1VI0aMCLH6XBmrsMHmueeeC2OyGhd/9rOfhdjEiRNDLOvrfdG8ZM2JLIUNunfccUedj/nCCy+E2B//+McQe++990Lsnnvuyds+55xz6pwHWqa2bduG2N57711rLOtKs1nPv1nNjFmyGhyzzJ8/v6hxhd5///0Qq5QrKZaCFQQAABBQIAAAgIACAQAABBXfg9CnT58QO+qoo0Is62JEBx54YIg99dRTedtZ30CXdR62S5cuNeZZX1nnxLLOQW+11VYhdsghh4TYyJEjQ2yFFVaoY3aoZAMGDMjbzurRKVbPnj1DLKsHIcsnn3xS5+MCNSn2/H2xvQUN7d133w2xnXfeufETqSdWEAAAQECBAAAAAgoEAAAQUCAAAICg4psUs5x77rkhNnbs2BD78MMPQ6yw4SqrkbFDhw71yK58shpuspoPDzrooBDbbrvtQizrm86yLlyC5qWcF8W64ooryrYvLNu++uqrENthhx1CbL/99guxk08+OW+7R48eZcurMWS9rjRHrCAAAICAAgEAAAQUCAAAIKBAAAAAQbNsUmzfvn2IjR49OsRatYr1T5s2bRokp8ayxx57hNj48eNDLOuKi3379g2xMWPG5G1n/c7Q/C1atCjEfvnLX4bYqFGjitpf1hVIs/aHZdfqq68eYoMGDQqxrG/snTt3bq1jmlvjYnPEqwEAAAgoEAAAQECBAAAAAgoEAAAQNMsmxSzt2rVr6hSazCabbBJid9xxR4gdeeSRIXbTTTflbZ966qnlSwz18rOf/SzE3n777RAbPnx43vZrr70Wxnz//fchljWuWFdeeWWIde/evc77Q8uz3HLLhdixxx4bYnfddVeIDR06NG/75ZdfDmNOOeWUEOvSpUuI7b///jXmWZOJEyfWOibr+XfFFVes8zErCSsIAAAgoEAAAAABBQIAAAgoEAAAQNBimhSR77DDDguxmTNnhtgZZ5yRt73uuuuGMQcccED5EkPRrrrqqhB76aWXQuyNN97I2546dWqdj9m5c+cQy2pIPOqoo+p8DCy7sq6u+Oyzz4ZYYYPu+++/H8YUfiW0JLVu3TrEsq68W6zCKzpmOfPMM0OsQ4cOdT5mJWEFAQAABBQIAAAgoEAAAAABBQIAAAhoUlyGDB48OMRGjhyZt3366aeHMTQpNo2shqsTTzwxxG677ba87QkTJoQxWV//nRW76KKLQqxbt2415gnUR9b8eu655/K2f/3rX4cxw4YNC7GsrzX/+uuv655cgU033TTE+vfvX7b9VxpWEAAAQECBAAAAAgoEAAAQUCAAAIDA3L3owb179/asBig0X0uXLq1xW5KWX764XlYzm+juvcuSWCNhTqMmzOnKkPU6tXjx4hC77777QuyTTz4JsayvmO7Zs2eI9erVK2/7N7/5TRizxhprhFglK2VOs4IAAAACCgQAABBQIAAAgIALJS3jWrVqVeM2ADQ1MwuxrAuJHXPMMUXtL6uXABGvBgAAIKBAAAAAAQUCAAAIKBAAAEBAgQAAAAIKBAAAEFAgAACAgAIBAAAEFAgAACCgQAAAAAEFAgAACCgQAABAQIEAAAACCgQAABCYuxc/2GympE8bLh0CVe+OAAAgAElEQVQ0c93dfY2mTqIUzGnUgjmNlqboOV1SgQAAAJYNnGIAAAABBQIAAAgoEAAAQECBAAAAAgoEAAAQUCAAAICAAgEAAAQUCAAAIKBAAAAAAQUCAAAIKBAAAEBAgQAAAAIKBAAAEFAgAACAgAIBAAAEFAgAACCgQAAAAAEFAgAACCgQAABAQIEAAAACCgQAABBQIAAAgIACAQAABBQIAAAgoEBoRsxsnJkNaOo8gHIys/vM7JKmzgMol5Yyp5u8QDCzb3N+lprZ/Jztw5swr7Zm9qGZTSnhPpeb2aI099lm9oqZbduAadaUy0Aze8PM5pjZNDO70syWa4pclkWVOK/NrLeZ/TXNYbqZDS7yfsea2ZL0fnPMbJKZ7d3Q+daQzxFm9mmazyNmtkpT5bIsqbQ5bWarmtm9ZjbTzL4ys4tKuG+lzenTzWxKmsvfzewnTZVLriYvENy9fdWPpKmS+uXE7i8cb2bLN1Jq50n6sg73uz99LGtKel3Sw1mDGuFxtJN0iqROkraTtJekMxr4mEhV2rw2szUlPS3pFkmrSdpQ0vMl7OKv6WNZVdI9kkaaWceM4zT04/iRksdwuKTOkhZJuqkhj4lEpc1pSTdIai2pm5LnuIFmdmQJ96+UOb29pMskHSBpFUn3SnrEzKwhj1uMJi8QapO+Kx9hZg+Y2VxJRxQu35jZ/+S+0zezrmY2Kq0sPzGzk0s85vqSDpF0TV3zdveFku6WtI6ZrZJWrGPN7AYz+z9JF6bHOtbM3jezr83sGTNbNyePPc3sAzP7xsz+IKnoCePut7j7K+6+0N2nSRouafu6Ph6UVxPM67MlPeXuD6RzYo67v19q3u6+RNKdklaUtF5Vjmb2KzObLulPaa77mtk/0pW0cWa2ac7j2NrM3jSzuWb2gKS2JaRwhKRH3X2cu38r6deSDjazFUt9LCivJpjTfSVd7e7z3f1jSXdJGlhq3hUwp3tIesvdJ7n7UiXFylpK3tw1qYovEFIHKHmB6yhpRE0DLVlGf1LSeEnrSNpd0jlmtlt6+05mNquW490k6ZeSvq9rwmbWVtIASVPcfXYa/omk9yStIelqM+sv6RxJ+6Wx15U8zqp3fA8pWcnoJGmapG1z9r9eOlG7FJnSjpLeqevjQYNozHm9naTZZvZauhz7mJl1LTXh9N3UIElzJf0rDXeV1F7JO7lfmFkfJU+qx0paXcmT72Nm1ib9f/FYGlst/fv+uY8zndfbVZPCDyX9o2rD3T+QtFTSBqU+FjSIxpzTpvw3TSZp02rG1pRHU8/ppyS1M7M+6e9koKSJ7j6z1MdSbs2lQBjn7k+4+1J3n1/L2O0kdXD3Iek7pY8k3SHpUEly95fdvdrKzMwOlrTY3Z+oY66HmdlsSZ9J2kzJf5gqU939Vndfkj6OEyQNcfcP3H2xpMslbWNm6yipjt9091HuvkjStZL+M2Hc/RN3X8Xdv6gtITM7TtKPJP2+jo8JDaPR5rWSJ7yjJZ2s5Envc0lhWbgGO6Tzerqk/pL2d/e56W2LJV2S5jVf0vGSbnH38elcvzMd10fJKpZLutHdF7n7g5ImVR0kHb+Ku79WTR7tJX1TEJsjaeUSHgsaTmPO6dGSzjOz9ma2gZI3ZKWsJFXKnJ4j6RFJf5O0QNL56fGaXGOdz6+vz0oY211St/Qfvspykl6q7Y5m1l7SlZL2KCm7fMPdfUA1txU+ju6Sbk5PH1RZquTJvEvueHdfambTSk3GzA5Scn5rN3f/v1LvjwbVKPM6NV/SX9x9oiSZ2aWSpptZ+3Spvjbj3H3nam6bkZ5Sy831cDPL7Xlpo+RdYhtJ09zdc277tMjHIEnfSupQEOug5N0fml5jzunBkm5U8q5/pqQHJB1UwvErZU6fIOlISb0kfSxpT0lPm9nm7j6jhP2UXXMpELxge57yK8XOOX//TNKH7r5JHY6zsZJ3V69Y0h/SRlLH9DxUH3cvZfJnKXwcn0m6yN3DUlx6fmvPnO1WSgqHopnZPpJulbSXu3N6ofI01ryWpH9mHK9wu66y5vWl7n514cB0+bhwHndT8ae/3pG0ec7+NlSyEvph0dmiITXanHb3WZJ+XrVtZtdI+ntd9pW1+4LthpzTm0t63N2r5vBT6amVH0t6tPiUy6+5nGIo9KakfSz5mMvakk7Nue1VSQvN7Cwza5ee/9nMzLYucr/dJG2R/pwg6Yv0719IkiUfGTyiTI/jj5IuMLNN0n2vkvYlSMm5uS3MbL/0HNkZSvoUimJmuytpdjmg6l0jKl5DzWspaeDqb2Y/MrPWki6Q9HLV6kHadHVhmR7HbZJOTs+pWroE3M/MVpI0TlIrMxtsZsunp/S2KmHf90na38x+ku7vN5JGuvt3Zcod5dVgc9rM1jez1dJ5tI+SPoIrcm5vLnN6vKS+ZtYj3fcekn6gCugZa64FwjAlzX6fKjkP9WDVDem5/L0lbSNpiqRZkoYqXZY0s50LlrSUe193n171I+lrSUvS7SVm1k7JR2JeL8eDcPeRSvoCRprZHCXv8vZIb5uh5JMUv5X0byWFy3+Oa2Y9LfkMb3VNir9W0ig0xv77WeW69lWgcQxTA8zr9P7PKpkTz0j6SsmSaW6h21XSK+V4EO7+uqSTlKxefS1pctWx3H2Bkr6c49LbDlTOu6T0ReJbM/txNfv+p5Kl5QfTx9FWycd5UZmGqYHmtJLz/+8oOYd/maRDCj6Z0yzmtJLi/RFJY9PHcp2kQTkrCk3G8k+boCZmtrOSf7hSPmsLVDQz6yHpXnf/aROnApQFc7o8KBAAAEDQXE8xAACABkSBAAAAAgoEAAAQlHQdhE6dOnmPHj0aKBU0dxMnTpzl7kV/FLMSMKdRE+Y0WppS5nRJBUKPHj00YcKEumWFFs/MSrl6WEVgTqMmzGm0NKXMaU4xAACAgAIBAAAEFAgAACCgQAAAAAEFAgAACCgQAABAQIEAAAACCgQAABCUdKEk1O67774LsUMPPTTEevbsmbd9/fXXN1hOAACUihUEAAAQUCAAAICAAgEAAAQUCAAAIKBJscymTZsWYk888USIrbDCCnnbF198cRiz6qqrli8xLDPeeuutENt1111DbNasWSE2fvz4EOvdu3d5EgPQrLCCAAAAAgoEAAAQUCAAAICAHoQmstZaa+Vtt2nTpokyQXM3aNCgvO177703jFm8eHGIbbjhhiHWuXPn8iUGoFljBQEAAAQUCAAAIKBAAAAAAQUCAAAIaFJsInvttVfe9korrdREmaC5GzNmTN52sQ2Jo0ePDrGuXbuWLzEAzRorCAAAIKBAAAAAAQUCAAAIKBAAAEBAk2KZ3XrrrSHWtm3bEDv99NMbIx20ML/4xS9CbMaMGXnbG220URjzzDPPhFiPHj3KlhfQkLK+ZfS2224LscmTJ4fYD37wgxA78MADQ2zbbbcNsTXWWKPYFFskVhAAAEBAgQAAAAIKBAAAEFAgAACAgCbFepg6dWqIDRs2LMRWXHHFEMu6sh1Qm5EjR4bYkiVL8rb//Oc/hzE0JKI5+fLLL/O2+/fvH8ZkPf8uv3x8SRs7dmyI3XXXXSHWu3fvEPv973+ft/3Tn/40JtuCsYIAAAACCgQAABBQIAAAgIACAQAABDQp1sPzzz8fYrNnzw6xq666qjHSQQtz5513hljW/DrkkEPytjfZZJM6H/OLL74IsRdeeKGo++6666552126dKlzHli2tWqV/9517ty5Ycwqq6wSYg888ECIZf2fOf/880NswoQJIfb444/nbdOkCAAAlnkUCAAAIKBAAAAAAQUCAAAIaFIs0ldffRVi11xzTYh17tw5xAYMGNAQKaGFmzNnTogVXjVRil9Tm3U1uayve7766qtD7OOPPw6xadOm1Zhnla5du+Ztr7TSSmFMp06dQuzUU08Nsayr2q233npF5YHmb6211srbzmoOLGwglLLn/qGHHhpiO+ywQ4jdcsstIXbrrbfmbf/kJz8JYw444IAQaylYQQAAAAEFAgAACCgQAABAQIEAAAACmhSLlNXk9cEHH4TYwQcfHGKFDTeSNH/+/LztxYsXhzErr7xyKSmihbn55puLGld4JcUnn3wyjMmalwsWLKhbYtUoppkx6//MK6+8EmK9evUKsazHxddYLxsOP/zwEMtqUhw0aFCIXXLJJUXd94033gixefPm1bjd0rGCAAAAAgoEAAAQUCAAAICAHoQMWeeZ7rnnnqLue+6554ZYVn9B4cU7ZsyYEcY8/fTTIbbaaqsVlQeal7vvvjvEpkyZUtR9f/e73+Vtjxo1KozJ6jfIuljM2WefHWLrrLNOUXkUY8SIESE2fPjwEHv33XdD7MYbbwyxa6+9tjyJoaKtuOKKIWZmITZ16tQQGzhwYFHHaNeuXYjdfvvtedtHHHFEUftqKVhBAAAAAQUCAAAIKBAAAEBAgQAAAAKaFDNcd911IfbCCy+E2C677BJiWd9C9+yzz4ZY1oU6Cn322WchRpNiy5TVpJr1zY1ZsuZroayLdQ0bNizEevbsWdQx62rrrbcOsX322SfEsv5v3XTTTSG23XbbhVjWRaHQvPXt2zfEHnrooRB78803Q+zyyy8PMXcPsT59+oTYUUcdVWyKLRIrCAAAIKBAAAAAAQUCAAAIKBAAAECwzDcpvv322yF22223FXXfrCt0zZo1K8ROOeWUWve19tprh1jnzp2LygPIteaaa4bYgw8+GGIN3ZBYrA022CDEspoqsxo5X3vttRCjSXHZcOCBBxYVu+qqq0Js0aJFIbasNyRmYQUBAAAEFAgAACCgQAAAAAEFAgAACFp0k2JWI8ro0aPztk866aQw5vPPPy9q/1kNMWPGjAmxyZMn17qv5ZeP/xRZXxOd9bW9bdu2rXX/aLnWWGONvO0TTzwxjNlpp50aK52SdenSJcSymiqzrq54xx13hBhfAY26aN26dVOnUHFYQQAAAAEFAgAACCgQAABAQIEAAACCFtOk+M0334TYAQccEGIvvvhi2Y650korlW1fWV/t3LVr1xDr1q1biN1+++0htvvuu5cnMVS8woa+rGa+5mb99ddv6hTQDP373/8Osayvds6y+uqrlzudZo8VBAAAEFAgAACAgAIBAAAEFAgAACBolk2KWQ2JZ599dogV05DYvn37ovbVoUOHEHvggQdCbPz48bUesz6yrrj4xhtvhBhNisuOjTfeuKlTKLusr00HavPEE0+EWNYVabOuPtu3b98Gyak5YwUBAAAEFAgAACCgQAAAAAEFAgAACCq+STHrK5uzmgizriZYjEsvvTTEzjzzzBD7/vvvQ+yyyy4r6hhmFmKbb7553vauu+4axvTr1y/EttpqqxDLaqAEmous/+NDhgwp6r4HHXRQudNBMzZ9+vSixg0cOLCBM2kZWEEAAAABBQIAAAgoEAAAQFDxPQgffvhhiNW130CSjjzyyLzt0047raj7jRgxIsS+/vrrou675557htjTTz9d1H2xbNhrr71C7OKLLw6xBQsWhNj999+ftz1o0KAwZtVVV61Hdg3r3XffDbGRI0eG2MorrxxiZ511VoPkhMqX9c2NQ4cOLeq+Bx98cLnTaZFYQQAAAAEFAgAACCgQAABAQIEAAACCim9SvOaaa+p83/XWWy/ECi9utNxyyxW1r5kzZxY17qijjgqxu+66q6j7Ytm12WabhVjWRYCGDx8eYueee27edlYT7+DBg0PsxBNPDLGsbwutq6xv0fvoo49C7JBDDilqf/379w+xXr16lZ4YWoSvvvoqxKZMmVLUfTt27FjmbFomVhAAAEBAgQAAAAIKBAAAEFAgAACAoKKaFLOujPXiiy8Wdd+2bduG2AMPPBBi3bt3Lz0xSZ9//nmItWvXLsSyGq5ataIOQ+myrhKY9c2gDz/8cN725MmTw5hTTz01xF5//fUQ69SpUykp5in8RtI///nPYUzhVR8lafXVVw+x66+/PsR22WWXOucGoHS8cgEAgIACAQAABBQIAAAgoEAAAABBRTUpLlq0KMS+//77ou771FNPhdi2225b75yqFF6tTsq+auKWW25ZtmNi2ZY1l+69994QK5ybWVcfLbZhsD7+8Ic/1Dqmc+fOIXbJJZeE2PHHH1+OlADUAysIAAAgoEAAAAABBQIAAAgoEAAAQFBRTYpZDUwzZsxogkyitddeu6gY0NgKvyo6q5Hx/PPPD7Err7wyxOrTuNi1a9e87axGw6yvsN5kk03qfEwADYcVBAAAEFAgAACAgAIBAAAEFAgAACCoqCZFAA2jV69eIZbVzJgVA7BsYgUBAAAEFAgAACCgQAAAAAEFAgAACGhSBAA0O2uttVaI9e7dO8QmTJgQYttvv32I7bDDDiH23HPP1TG7loEVBAAAEFAgAACAgAIBAAAE9CAAAJqd1VZbLcSefvrpEOvSpUuILViwIMQGDx5cnsRaEFYQAABAQIEAAAACCgQAABBQIAAAgIAmRQBAi7DGGmuE2KJFi5ogk5aBFQQAABBQIAAAgIACAQAABBQIAAAgoEAAAAABBQIAAAgoEAAAQECBAAAAAgoEAAAQmLsXP9hspqRPGy4dNHPd3T1eyqyCMadRC+Y0Wpqi53RJBQIAAFg2cIoBAAAEFAgAACCgQAAAAAEFAgAACCgQAABAQIEAAAACCgQAABBQIAAAgIACAQAABBQIAAAgoEAAAAABBQIAAAgoEAAAQECBAAAAAgoEAAAQUCAAAICAAgEAAAQUCAAAIKBAAAAAAQUCAAAIKBAAAEBAgQAAAAIKBAAAEFAgNCNmdp+ZXdLUeQDlZGbjzGxAU+cBlEtLea5u8gLBzL7N+VlqZvNztg9vgnzOM7N3zGyumX1sZmeWcN9jzWxJmvscM5tkZns3ZL415HJRwe92fprbqk2Rz7KmAuf15Wa2qCCvbnW472wze8XMtm3onKvJ5XAz+8DMvjGzGWZ2l5m1b4pcljUVOKfPMrNP0ufaz83sWjNbvsj78lxdhCYvENy9fdWPpKmS+uXE7i8cX+wEqKcjJK0iaR9JZ5hZ/xLu+9f0sawq6R5JI82sY+Gghn4c7n5Zwe/2Wkl/cfevG/K4SFTovL4/Ny93n1rqfSWtKel1SQ9nDWqEx/FXSdu7e0dJ60taQdJvGviYUEXO6UclbeHuHSRtLqm3pF+UcH+eq2vR5AVCbdJ3LyPM7AEzmyvpiMLlGzP7HzObkrPd1cxGmdnMtMI8udjjuftV7j7J3Ze4+3uSnpC0fal5u/sSSXdKWlHSelU5mtmvzGy6pD+lue5rZv9I35mNM7NNcx7H1mb2Zrqa8YCktqXmke7HJB0p6e663B/l19jzulzcfaGSebSOma2SvhMba2Y3mNn/SbowzfVYM3vfzL42s2fMbN2cx7FnzirAHyRZCcef6u6zckJLlRQKaGJN8Fz9L3f/JidUp7nAc3X1Kr5ASB0gabikjpJG1DTQzJaT9KSk8ZLWkbS7pHPMbLf09p3MbFYNu8jdVytJO0h6p9SE06pzkKS5kv6VhrtKai+pm6RfmFkfJZPvWEmrK5mkj5lZGzNrK+mxNLZa+vf9cx9nOlG3KyKdXZRUyaNKfRxoUI09r/c3s/8zs7fN7IS6JJzOywGSprj77DT8E0nvSVpD0tXpits5kvZLY68reZwyszUlPSTpPEmdJE2TtG3O/tdL53WXGnLYycy+kTRH0r6Srq/LY0GDaNQ5bWZHpsXITEk/lHRbqQnzXF295lIgjHP3J9x9qbvPr2XsdpI6uPsQd1/o7h9JukPSoZLk7i+7e6cij3uZpMVKlp+KtYOZzZY0XVJ/Sfu7+9z0tsWSLknzmi/peEm3uPv4dMXiznRcHyWrFi7pRndf5O4PSppUdZB0/Cru/loROR0t6c/u/l0JjwMNrzHn9QOSNlHygn2ipN+Y2cEl5HpYOq8/k7SZkheCKlPd/dZ0Ts6XdIKkIe7+gbsvlnS5pG3MbB1JfSW96e6j3H2RkuXUmVU7cvdP0nn9RXWJpI+1o6R1Jf1OyXI3KkOjPle7+73uvrKkjSUNlfRVCbnyXF2LxjjvWQ6flTC2u6Ru6T98leUkvVTKAc3sNCUT9afpsmqxxrn7ztXcNqNgX90lHW5mZ+TE2iippttImubunnPbpyXkIUkys5UkHSRpr1LviwbXaPPa3XNXwcaZ2Y1KnhRHFnn84e4+oJrbCh9Hd0k3p6cPqixV8q6sS+54d19qZtOKzCGPu08zs+eVvGPdpi77QNk1+nO1JLn7B2b2gaSbJP1vkXfjuboWzaVA8ILteUrOF1XpnPP3zyR96O6b1PVgZna8pLMk7VjTO5k6KHwcn0m61N2vzshhNyVPqLm6qfTTHf0lzZA0rsT7oeE16rzOOHbR5/6L2FeuzyRd5O5hiTk9b7tnznYrxXleiuUl/aAe90d5NeWcLudc4LlazecUQ6E3Je1jZqua2dqSTs257VVJCy35CEy79PzPZma2dTE7NrOjJV0qaXd3n5Jx+zgzu7AMj0FKzpedbGZ9LNHezPqlleQ4Sa3MbLCZLZ8uB29Vh2McLenuguoWlakh5/X+ljQVmiUfURys5Fxp1e3TzOyIMj2OP0q6wMw2Sfe9iv33k0BPStrCzPZLz/2eoeS0R1HM7AhLGx7NrIeS04B/KVPeKL+GnNPHpT0tMrMfSvqlcuYCz9X111wLhGFKmqI+lTRa0oNVN6TnPPdWsuQ4RdIsJeemOkiSme1csKRV6HIlTSgT7b+fS70p5/aukl4px4Nw99clnSTpVklfS5qs5COWcvcFSs7zHpfedqCSj/UofRzLpbn9uLr9W/I59x0l3VuOfNHghqnh5vVhkj5W0oh1t6TLqz6aZmbtlDRGvV6OB+HuIyX9XsnHxuZI+qekPdLbZkg6RNJvJf1byTut/xzXzHqm87q6JsXNJL1mZvOUPDG/o6TnAZVpmBpuTu8o6e10Ljwp6XFJF+XcznN1PVkFFSsVL33Hcq+7/7SJUwHKxsx2ljTI3Y9s6lyAcuC5ujwoEAAAQNBcTzEAAIAGRIEAAAACCgQAABCUdB2ETp06eY8ePRooFTR3EydOnOXuRX9krRIwp1ET5jRamlLmdEkFQo8ePTRhwoS6ZYUWz8xKvnpYU2NOoybMabQ0pcxpTjEAAICAAgEAAAQUCAAAIKBAAAAAAQUCAAAIKBAAAEBAgQAAAAIKBAAAEFAgAACAgAIBAAAEFAgAACCgQAAAAAEFAgAACEr6NkcAy57JkyeH2AknnBBihx12WN72cccd12A5AWh4rCAAAICAAgEAAAQUCAAAIKAHAcB/ZPUb7LPPPiH28ccfh9iUKVPytulBAJo3VhAAAEBAgQAAAAIKBAAAEFAgAACAgCZFYBn1hz/8IcSuv/76EJs6dWpR++vevXu9cwJQOVhBAAAAAQUCAAAIKBAAAEBAgQAAAIJlrklx9uzZedsffvhhGDN8+PCi9pXV0GVmdcqrc+fOIfbqq6+GGI1gqIvFixeH2Lvvvhtin376aYhlzekNN9wwxO677746ZgegErGCAAAAAgoEAAAQUCAAAICAAgEAAAQtukkxq2lqyJAhedsffPBBnfef1by1+eabh9iiRYtC7L333svbnjFjRhgzffr0EKNJEXXxxz/+McRuv/32Ou+vU6dOIda1a9c67w/Nx8yZM0PsggsuCLHXXnstxDbZZJMQO+WUU2o9Zs+ePUOsS5cutd4P9cMKAgAACCgQAABAQIEAAAACCgQAABC0mCbFrKsfnnTSSSH23Xff5W2vttpqYcyBBx4YYlnNhzvuuGOIZTURZl3Fbt11183bnj9/fhiT9Zi23XbbEANyffHFFyF2xx13hJi7FxXL8tvf/rb0xFBRnnzyyRA777zz8razvuo7q+n6+++/D7Gsq8O+9dZbITZq1KgQW7JkSd52mzZtwpjWrVuHWNZz92GHHRZiWfr06RNiq666alH3balYQQAAAAEFAgAACCgQAABAQIEAAACCZtmkWNhoKGVfFW7rrbcOsQsvvDBve/vttw9jVlhhhXpkF2U1IBbztdAHH3xwWfPAsiHrK5v/+c9/hlixX02+7777hthWW21VemKoKFdccUWIvfPOO3nbm266aRiTdQXDrCbu3XffPcSyGhyzGgEXLlyYt/3666+HMaNHjw6xrKbKrOfROXPmhFjW1UHPPffcvO1+/fqFMRtvvHGItRSsIAAAgIACAQAABBQIAAAgoEAAAABBs2xSXHHFFUPshRdeaIJMinPttdeGWGGj5QYbbBDGZH01KlCblVdeOcSyGrBmzZpV1P5effXVEJs8eXKIZTW0oXJlXV1zwIABedu9evUKY4YNG9ZAGVVvp512CrHCBsLqfP311yFW2IwpSQ899FCIDRkyJG/77rvvDmNuvvnmEMvKtzliBQEAAAQUCAAAIKBAAAAAQbPsQahk48ePD7Grr7661vtlffPk6quvXpacsGzJ6gXYf//9Qyzr4mJZsnoVbrnllqJiqFxZ/QVjxozJ227btm1jpdNgsi7EtM0224TYmmuuGWKFFx179NFHw5irrroqxN54440QO+qoo0Ks0p/jWUEAAAABBQIAAAgoEAAAQECBAAAAApoU62Hp0qUhVtjkI2V/+2THjh3ztnfZZZfyJQYUKPwWU6n4JsUsTzzxRIidcMIJedtZ3/CHypbV0NfcPfnkkyF24oknhtjs2bNDbN68ebXuP+ubJp999tkQ+9GPfhRiu+22W637b0qsIAAAgIACAQAABBQIAAAgoEAAAAABTYr1kPVtaBdffHFR9y28+lZWAwtQLuuuu26InXbaaSF23XXXFbW/zz//PMT23XffvO3Cq9ABTeHLL78Msaz5m6XweXnatGlhzOWXXx5ihU3oUuU3JGZhBQEAAAQUCAAAIKBAAAAAAQUCAAAIaFKsh6wrdGXp1q1biCk77gcAAAWiSURBVB199NHlTgcoSVZDbe/evUOs8AqJUvbVQadPn563feqpp4YxAwcODLEtttiixjyB+hgwYECIjR07NsQeeuihECtsJt9yyy3DmM6dO9c9uQrHCgIAAAgoEAAAQECBAAAAAgoEAAAQ0KRYpEmTJoVY1lfemlmInXPOOSHWtm3b8iQG1FHW1d4OO+ywEBs5cmSIvfTSSyE2Z86cvO2bb745jMlqBPvHP/4RYmussUaIAXXRunXrELv++utDbOWVVw6xk046KW97q622CmPuueeeEGvfvn0pKVYsVhAAAEBAgQAAAAIKBAAAEFAgAACAgCbFDPPmzQuxSy65JMTcPcSyvtLzF7/4RVnyAprCqFGjQmzo0KEhVsw8L7zaoiQtXLiwbokBdbT66quH2A033BBi/fr1y9s+6KCDwpjCrzmXpGuvvTbEsq7CWOlYQQAAAAEFAgAACCgQAABAQIEAAAACmhQz3HXXXSGW9dXOK6ywQogdc8wxDZITUEl+9KMfNXUKQFktv3x8Odxrr73ytidMmBDG9O3bN8Quu+yyEMtq7K30K4ayggAAAAIKBAAAEFAgAACAYJnvQfjwww9D7IILLijqvlnf0pj1bXhAXbz88stFjdtpp50aNI8//elPITZkyJAQy7pwWF3GAJWi8KJ5WT0IX3/9dYhlXVxs/PjxIfbZZ5/VI7uGxwoCAAAIKBAAAEBAgQAAAAIKBAAAECxzTYqFTVJXXnllGPPtt98Wta+sC2QAdfHFF1+E2H777RdiO+64Y4h99dVXdTrm448/HmJZjZEzZswIscWLF4eYmeVtb7HFFkUds3PnzjXmCZRb1v+34cOHh1jhxY0++uijovbfrl27ENt7772LzK5ysIIAAAACCgQAABBQIAAAgIACAQAABMtck+LDDz+ct3333XcXdb8BAwaEWJ8+fcqREqAlS5aE2Ny5c0Ms61tFn3rqqbLlkXWlw8LmQ0nq0KFDiF199dV52/369Qtj1l577XpkB9TstddeC7ERI0aE2J133hlic+bMqdMxV1lllRC79tprQ2zgwIF12n9TYgUBAAAEFAgAACCgQAAAAAEFAgAACJa5JsXJkyfX6X4XXnhhWfMobJw55JBDyrp/NC/LLbdciGU1An7zzTcNmse6664bYltuuWWInXbaaSG2yy67NEhOwH333RdiL774YoiNHDkyxLKafbP88Ic/DLEf//jHedvrrbdeGHP88ceHWKdOnYo6ZqVjBQEAAAQUCAAAIKBAAAAAAQUCAAAIlrkmxQkTJtQ65qKLLgqxbt26hdiCBQtC7JFHHgmxyy67LMRuvPHGWvPAsqNLly4h9uijj4bYpEmTat3XDTfcEGI7/397d4ijSBBGARgOgERjCYIg8IxDo7kECgOGIyAQHIggUCQEj96EK9Rq8ki6J5sZWPr7XL9UyC9KvFRX6K+vyIbDYWSLxaLy9+G3Pfv0+el0imw8HkfW7/cje/Yvn71eL7LBYFB3xI/kBAEACAoCABAUBAAgKAgAQGjcJcXj8Vi55n6/R3a9XiObz+eR3W63yFarVWSTyaRyDprt2R6ps29cNOTTdDqdyLbb7QsmaRYnCABAUBAAgKAgAABBQQAAQuMuKc5ms4fn/X4fa3a7Xa2slBLZs09/LpfL74wIAC/nBAEACAoCABAUBAAgNO4OwmazeXg+HA6x5nK5RDYajSJbr9eRTafTf5gOAN6DEwQAICgIAEBQEACAoCAAAKFxlxS73e7D8/l8ftEkAPC+nCAAAEFBAACCggAABAUBAAgKAgAQFAQAICgIAEBQEACAoCAAAKFdSqm/uN3+02q1bj83Dv+5XimlW73sfdjTVLCn+TS19/S3CgIA0AxeMQAAQUEAAIKCAAAEBQEACAoCABAUBAAgKAgAQFAQAICgIAAA4S+rdMrsqtGHJAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0xf16aa58>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Plot some of the correct and misclassified examples\n",
    "cls_pred = sess.run(cls_prediction, feed_dict=feed_dict_test)\n",
    "cls_true = np.argmax(y_test[:1000], axis=1)\n",
    "plot_images(x_test, cls_true, cls_pred, title='Correct Examples')\n",
    "plot_example_errors(x_test[:1000], cls_true, cls_pred, title='Misclassified Examples')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "After we finished, we have to close the __`session`__ to free the memory. We could have also used:\n",
    "```python\n",
    "with tf.Session as sess:\n",
    "    ...\n",
    "```\n",
    "\n",
    "Please check our [Graph_and_Session](https://github.com/easy-tensorflow/easy-tensorflow/blob/master/1_TensorFlow_Basics/Tutorials/1_Graph_and_Session.ipynb) tutorial if you do not know the differences between these two implementations.\n",
    "            "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "sess.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Thanks for reading! If you have any question or doubt, feel free to leave a comment in our [website](http://easy-tensorflow.com/)."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}
